Chapter 7 · Large language models
Emergent abilities
Some skills seem to appear suddenly as models grow; whether they are truly sudden is debated.
Chapter 7 · Large language models
Some skills seem to appear suddenly as models grow; whether they are truly sudden is debated.
All-or-nothing score
Score by getting the whole answer exactly right: it looks like a sudden jump.
Both lines come from the same toy model. Only the way of scoring is different.
Researchers tested many sizes of model on a task. Small ones scored near zero, then bigger ones suddenly scored well. People called this an emergent ability.
Toy numbers, invented for teaching. Per-token skill rises smoothly with model scale in this toy. Does the curve still look like a jump?
All-or-nothing, long answer: near zero, then a sharp rise. The smaller the number above, the more sudden the jump looks. This toy shows that a metric can create the look of a jump. It does not prove that every real jump is an illusion.
What does an emergent ability look like in a chart?
Why might all-or-nothing scoring create a jump?
What is the best conclusion about emergent abilities?