Chapter 7 · Large language models
Tokens, limits and cost
Usage is counted in input and output tokens; long prompts and long answers cost more and take longer.
Chapter 7 · Large language models
Usage is counted in input and output tokens; long prompts and long answers cost more and take longer.
Your prompt goes in. The meter counts input tokens.
Like a taxi meter, usage ticks per token: the ones you send and the ones you get back.
You met tokens earlier. Services count usage in tokens: input tokens (everything you send) and output tokens (everything the model writes).
Type a prompt. Prices are blank-slate numbers you choose, per million tokens, in any currency. The starting numbers are only examples, not real prices. Token counts come from a simple teaching tokeniser.
Assumes each turn adds a message the size of your prompt, a reply of the answer length, and a 40-token system message. Prices are yours; none are real.
What does a service usually count for billing?
Why does a long chat get costlier each turn?
What mostly decides how long the model takes to write an answer?