Chapter 7 · Large language models
Base models vs chat models
A base model just continues text; a chat model has been tuned to follow instructions and converse.
Preparing base models vs chat models
Chapter 7 · Large language models
A base model just continues text; a chat model has been tuned to follow instructions and converse.
The same prompt goes to two models.
A base model predicts what text comes next. A chat model has been taught to help.
You know pretraining teaches a model to predict the next token over huge amounts of text. The result is a base model. It is a very good text continuer.
Each round shows one prompt and two replies written for this lesson. Tap the reply you think came from the base model.
Prompt: What is the capital of France?
Score: 0 right out of 0 answered.
You give a base model a question. What may it do?
What turns a base model into a chat model?
Underneath, what do both kinds of model still do?