Chapter 8 · Shaping models: tuning and alignment
Fine-tuning
Fine-tuning continues training a pretrained model on a smaller, focused dataset.
Chapter 8 · Shaping models: tuning and alignment
Fine-tuning continues training a pretrained model on a smaller, focused dataset.
A pretrained model already knows a lot: it is a generalist.
Fine-tuning continues training, but on a small, focused set of examples.
You know pretraining is slow and costly. Fine-tuning does not start from nothing. It takes a model that already knows language and carries on training it.
The replies are written by hand to show the idea. Move the steps and watch the support answer change style. Then pick the small dataset and push the steps past 70 to see the traps.
Style is sinking in. Some sentences already follow the examples.
Question 1: “My parcel is late.”
Question 2, not in the training data: “Why is the sky blue?”
How is fine-tuning different from pretraining?
What mostly changes when you fine-tune on support replies?
A tiny dataset is trained for a very long time. What is likely?