Chapter 8 · Shaping models: tuning and alignment
Synthetic data
Models can be trained on data written by other models; it fills gaps but can copy their mistakes.
Chapter 8 · Shaping models: tuning and alignment
Models can be trained on data written by other models; it fills gaps but can copy their mistakes.
A teacher model writes practice examples…
Sometimes there is too little real data. A teacher model can write extra examples, but a student only learns as well as those examples are right.
Real data can be scarce, private or costly to label. Synthetic data is data written by a model or program instead of collected from the world. It can fill gaps and cover rare cases.
The truth: numbers from 5 up are Pass. The teacher writes examples and the student learns a cut-off from them. Then it is tested on real numbers.
Dots above the line are labelled Pass by the teacher, below are Fail. Pink dots are wrong labels.
The student has copied the teacher's mistake. Try raising the checker. Simplified teaching version with one number and a one-line rule.
A teacher model keeps making the same mistake. What happens to a student trained on its examples?
Which step helps most before training on synthetic data?
Why keep real data in the mix?