Chapter 3 · Neural networks1989
Why networks can learn almost anything
With enough neurons, a network can approximate any smooth function: little bumps add up to any shape.
Chapter 3 · Neural networks1989
With enough neurons, a network can approximate any smooth function: little bumps add up to any shape.
1 bump
Each pair of neurons makes one small bump. Lay more bumps side by side, each as tall as the curve at that spot, and their total follows the curve.
A single neuron with a squashing function makes a smooth step: low on one side, high on the other.
Drag the dots up and down to bend the dashed target, or pick a preset. Then raise the number of bumps and watch the purple line close in. Each bump costs two neurons.
A teaching version: the bumps are placed by a simple rule instead of being learned. A real network has to find good weights by training.
Why does adding more bumps help?
The theorem says a network with enough neurons…
What do two neurons make together in this lesson?