Chapter 3 · Neural networks
What weights really are
Weights are the learned numbers inside a model; together they store everything it knows.
Chapter 3 · Neural networks
Weights are the learned numbers inside a model; together they store everything it knows.
After 0 training steps: 54% of dots right
Each cell is one weight, a plain number. At the start the network guesses badly. After training, the same 17 numbers sort the dots almost perfectly.
Open any neural network and you find the same thing: a big list of numbers called weights (and biases). A neuron multiplies its inputs by weights, adds them up and passes the result on.
This network was trained to separate violet dots from coral ones in four corners. Tap a weight to edit it, or use the buttons to damage many at once. Then press Train.
Working well: the weights hold the pattern.
Tap any cell to edit one weight.
A teaching toy: 17 weights, one tiny pattern, trained live in your browser. Real models have billions of weights, but each one is just a number like these. Violet cells are positive, coral cells negative.
Where does a neural network keep what it has learned?
You set many weights to zero. What most likely happens?
What does freezing a weight do during further training?