How does a Deep Neural Network learn?

Greetings! Some links on this site are affiliate links. That means that, if you choose to make a purchase, The Click Reader may earn a small commission at no extra cost to you. We greatly appreciate your support!


The learning process of a deep neural network is also very similar to the learning process of a neuron. It goes through multiple finite iterations where data is feed-forwarded into the network and then, weights are adjusted using back-propagation (gradient descent).

Learning in Deep Neural Network

First of all, the input data is fed into the network through the input layer and all inputs are assigned with random weights at the start of training.

Then, the inputs, along with their respective weights, are passed into the hidden layers where each neuron applies an activation function on the linear combination of its inputs. Finally, the output layer outputs the probability of each label in the training set and loss is calculated using a loss function $J(w)$.

After the loss is calculated, the deep neural network back-propagates and gradient descent is performed to change the values of weights of all inputs for all neurons in the network to decrease the output loss. The gradient descent algorithm is still the same as we had studied before:

$$ (\textbf{w}^{(layer)}_{neuron})_{n+1} = (\textbf{w}^{(layer)}_{neuron})_n – \alpha\nabla J((\textbf{w}^{(layer)}_{neuron})_n)$$

This entire process is iterated multiple times until the loss of the neural network reaches a certain threshold. Once the model reaches the threshold, training is stopped.

How does a Deep Neural Network learn?How does a Deep Neural Network learn?

Do you want to learn Python, Data Science, and Machine Learning while getting certified? Here are some best selling Datacamp courses that we recommend you enroll in:

  1. Introduction to Python (Free Course) - 1,000,000+ students already enrolled!
  2. Introduction to Data Science  in Python- 400,000+ students already enrolled!
  3. Introduction to TensorFlow for Deep Learning with Python - 90,000+ students already enrolled!
  4. Data Science and Machine Learning Bootcamp with R - 70,000+ students already enrolled!

Leave a Comment