Chapter 7 · Large language models
Temperature
Temperature reshapes the next-token odds: low is safe and repetitive, high is creative and risky.
Chapter 7 · Large language models
Temperature reshapes the next-token odds: low is safe and repetitive, high is creative and risky.
Temperature 0.2
Very safe: almost always the top word
Same model, same word. Only the temperature changes. Low makes the top word tower. High lets the rest catch up.
After “the”, a model gives each next word a probability. You saw that in softmax and next-token prediction.
Turn the dial, then press Write again for another sentence. The same seed gives the same sentence.
Balanced: the model's own odds
Try 0.1 and press Write again: the sentence hardly changes. Then try 2.5. The tiny model learned from a few made-up sentences, so its writing is silly on purpose.
You lower the temperature. The favourite word's share…
Which job suits a low temperature?
What does temperature actually change?