Chapter 6 · The Transformer
The context window
The context window is how many tokens the model can see at once; anything beyond it is invisible.
Preparing context window: what the model can see
Chapter 6 · The Transformer
The context window is how many tokens the model can see at once; anything beyond it is invisible.
The window sees the first words.
The window is a fixed number of tokens. When the text grows, the oldest tokens slide out of view.
You met tokens before. A model reads your text as a row of them, not as whole pages.
Edit the text, then drag the window size. The question at the end only works if the answer is still inside.
23 tokens at the start are crossed out. The model cannot see them. Keeping the last tokens is a simplification; some apps shorten the oldest part instead.
What is the context window?
A chat grows longer than the window. What happens to the oldest tokens?
What shares the window?