Chapter 7 · Large language models
Small language models
Small models, trained on carefully chosen data, can do focused jobs well while running cheaply on ordinary devices.
Chapter 7 · Large language models
Small models, trained on carefully chosen data, can do focused jobs well while running cheaply on ordinary devices.
Two models, two jobs…
A bigger model can do more, but every answer costs more time and money. Many jobs do not need that much.
A model with more parameters usually knows and can do more. It also needs more memory, more power and more time for each answer.
Tick the jobs you need done. See which ones a small model can handle, and what training it for one area changes.
A small answer costs 1 unit and takes 1 time unit; a big one costs 20 and takes 6. These are made-up teaching units. A simplified chooser: real choices also depend on quality tests, privacy and the device.
What helps a small model do a focused job well?
Why might a company run a small model on the user's own phone?
A task needs long multi-step reasoning. What is the sensible choice?