Chapter 2 · Machine learning basics
Overfitting and underfitting
A model that memorises noise fails on new data; one too simple misses the pattern. Aim for just right.
Chapter 2 · Machine learning basics
A model that memorises noise fails on new data; one too simple misses the pattern. Aim for just right.
Too stiffA straight line misses the wave. It is wrong on the dots and on new data too.
The dots follow a wave plus a little random noise. The green dashed line is the true wave the model cannot see.
Training means adjusting the model until its curve passes close to the training dots.
Violet dots are training data. Coral rings are test data the model never saw. Drag the slider from 1 to 9 and watch the two errors.
UnderfitUnderfitting: too simple to catch the pattern. Both errors are high.
A simplified teaching version: ten training dots, forty test dots, and polynomial curves. The sweet spot moves a little with each new set of dots, but the shape stays.
A curve touches every training dot but does badly on test dots. What is this?
As you make a model more flexible, training error usually…
Which number should you watch to pick the best complexity?