Chapter 13 · Reasoning models2025
Reinforcement learning from verifiable rewards
Training on problems with checkable answers (maths, code) teaches models to reason longer and better.
Chapter 13 · Reasoning models2025
Training on problems with checkable answers (maths, code) teaches models to reason longer and better.
17 × 6 = ?True answer: 102 (the checker knows)
A model is asked a question with one right answer.
No person marks these attempts. A checker that knows the true answer ticks or crosses each one, and the model drifts towards whatever earned ticks.
Some tasks have an answer a program can verify: a maths sum, a puzzle, code that must pass its tests. The check is exact and cheap.
The toy solver has four ways to tackle sums. Press Run rounds and watch which habit wins. Then switch the reward to the judge that only likes long answers.
Press Run 1 round to start.
A teaching toy: four fixed habits with made-up success rates stand in for a language model, and the update is a simplified policy-gradient step. The checker really multiplies.
What makes a reward 'verifiable'?
What happened under the length-loving judge?
Which tasks suit this training best?