Chapter 7 · Large language models
System prompts and chat roles
Chats are turned into one token stream with roles: system sets the rules, user asks, assistant answers.
Preparing system prompts and chat roles
Chapter 7 · Large language models
Chats are turned into one token stream with roles: system sets the rules, user asks, assistant answers.
A chat looks like separate coloured messages.
The chat window is for you. The model only ever sees one flat run of tokens.
A chat has messages with a role. System sets the rules and the style. User is you, asking. Assistant is the model, answering.
Pick a system message and a question. The reply is written by hand to show the effect. Then add a follow-up and watch the stream grow.
Rough size: about 44 tokens, and it all goes in again on the next turn.
What does the system message do?
How does a model 'remember' earlier turns?
Are role tags like [USER] kept the same by every model?