Neural Networks: Supervised Learning

Feed Forward Neural Network with Back Propagation Training (Multi-layer perceptron)

Artificial Neural Networks (ANNs) are computational models representing structures and processes within real biological systems (animal/human brains). Similarly to real analogs ANNs are able to solve various tasks through the adaptation to external environment. One of the widely used concepts of such adaptation is supervised learning which assumes presence of training data - set of predefined samples. Each sample contain pair of numeric arrays - inputs and corresponding desired outputs. Thus, the training data forms expectations around the model. And model adjusts itself to fit the sample data by learning. Later on adjusted (pre-trained) model is used to calculate outputs for the fresh inputs, which mimics (or means) using previous experience for solving current task.

On very high level ANN can be described as a black box which implements a function:

outputs = F(inputs, internal parameters)

Outputs are calculated as functional dependency from the inputs provided and set of internal parameters. Function is pre-defined, parameters could change. And the main goal of learning is to find such parameters, so the difference between outputs produced by model and desired ones is minimal across entire training data set.

There are many various different structures and implementations exists, multi-layer perceptron with backpropagation training algorithm is the one of simplest and well-known.