Chapter 3 · Neural networks1986
Backpropagation
Backprop sends the error backwards, telling every weight how much it contributed so it can adjust.
Preparing backpropagation
Chapter 3 · Neural networks1986
Backprop sends the error backwards, telling every weight how much it contributed so it can adjust.
Input goes in
Violet is the forward pass making a guess. Coral is the error travelling backwards, telling each weight how much it caused the mistake.
Run a forward pass. The network gives an answer, say 0.62, when the right answer is 1.
Press the button to move through Forward, Error, Gradients and Update. These are the real gradients of this tiny network. Repeat and watch the loss fall.
What does backpropagation give each weight?
Which way is a weight moved?
Why is one backward sweep so useful?