Chapter 2 · Machine learning basics
Bias and variance
Bias is being consistently off; variance is being jumpy. Good models balance both.
Chapter 2 · Machine learning basics
Bias is being consistently off; variance is being jumpy. Good models balance both.
The bullseye is the right answer. Each dart is one model trained on its own random dataset.
Your training data is only one sample of the world. Collect another sample and you would get a slightly different model.
Pick how flexible the model is, then throw. Each dart trains a fresh curve on a new random dataset. Orange darts were flung off the board.
The magenta ring is the average of all darts. Bias is how far that ring sits from the bullseye. Variance is how far darts stray from the ring. A simplified teaching version with curves trained on ten noisy dots.
Darts land close together but far from the bullseye. This is…
A very wiggly model gives a very different answer on every new dataset. This is…
What usually happens to bias when a model becomes more flexible?