Chapter 9 · Talking to models: prompting and context
Long context and 'lost in the middle'
Models can accept very long inputs but may miss details buried in the middle.
Chapter 9 · Talking to models: prompting and context
Models can accept very long inputs but may miss details buried in the middle.
Edges stand out. Easy to notice.
Illustrative curve. Start and end are noticed most; the middle of a long input is noticed least.
Many models accept very long inputs: whole reports or books. A big context window means it fits. It does not mean every part is used equally well.
The orange cell is your hidden fact. Slide it through the text and stretch the length. The toy model shows how likely it is to be noticed.
Shaky. It might be missed.
This is an invented teaching curve, not a measurement of any real model.
A model accepts a 200-page input. Does it use every page equally well?
Where is it wise to put a vital fact?
What else can help a vital fact survive a long input?