Chapter 7 · Large language models
Knowledge cutoff
A model only knows what was in its training data, up to a cutoff date.
Chapter 7 · Large language models
A model only knows what was in its training data, up to a cutoff date.
The model reads text month by month. Each filled square is a month it has seen.
After training ends the model is frozen. The world keeps going, but the model does not.
You know that pretraining reads huge amounts of text. That pile was collected up to a certain date. We call that date the knowledge cutoff.
The months are made up for this lesson. Drag the sliders. Each question shows what a model with that cutoff could answer. Then turn on fresh text.
15 months have passed since the cutoff. The model does not know about any of them.
What is a knowledge cutoff?
You ask for the newest version of something a year after the cutoff. What is likely?
How can a model answer a question about last week?