How a Neural Network works?- An Example

Let’s take an example using the MNIST dataset. In the MNIST dataset, each input has a pixel size of 28 x 28. Each pixel has its corresponding numerical value ranging from 0 to 255.

Here, we are taking an example of the digit 5. Thus, we will have 28 x 28 input features (784 neurons in the input layer of the neural network) and also need to normalize the values to bring them between 0 and 1.

The neural network architecture is structured as follows. We can clearly see the input layer, hidden layers, and output layer.

  • Input layer: Contains 784 features, represented as x1 through x784.
  • Hidden layers: At each hidden neuron, a weighted sum is calculated (z = ∑ wixi + b). This result is then passed through an activation function, such as ReLU (Rectified Linear Unit, max(0, x)).
  • Output layer: Designed for a multi-class problem (e.g., MNIST dataset), mapping to classes 0 through 9, typically utilizing a softmax function.

At a neuron in a hidden layer, we perform a weighted sum and apply an activation function. This output is then passed forward to another neuron in the subsequent hidden layer, and so on.

Finally, at the output layer, we compute the final probabilities to solve the target classification problem.

Simple demo example:

error: