Chapter 7 · Large language models
Next-token prediction
An LLM writes by predicting the next token, adding it, and repeating.
Preparing next-token prediction
Chapter 7 · Large language models
An LLM writes by predicting the next token, adding it, and repeating.
At every step the model scores every possible next token, picks one, adds it, and starts again with the longer text.
The model reads all the tokens written so far, including your prompt.
Click a word to add it, or press Generate to let the model sample. Lower temperature makes it play safe; higher makes it adventurous.
This toy model only knows which word follows which in a few dozen sentences. Real LLMs look at the whole context with billions of weights, but the writing loop is the same.
How does a language model write a paragraph?
What does a higher temperature do?
Before choosing, the model gives a probability to…