Chapter 11 · Retrieval and memory
Chunking documents
Long documents are split into chunks so the most relevant pieces can be found and fitted into context.
Chapter 11 · Retrieval and memory
Long documents are split into chunks so the most relevant pieces can be found and fitted into context.
A long page is too big to search or paste whole.
Overlap is a safety net: the end of one piece is repeated at the start of the next.
A manual may have thousands of words. A prompt has limited room, and a search that returns the whole manual is not much help.
Drag the size and overlap. Purple marks start a new chunk and coral words are repeated in two chunks. Then pick a fact and see if some chunk holds all of it.
Counting in words keeps this simple. Real systems count tokens and often cut at sentence or heading ends instead of at a fixed number.
Why split a long document into chunks?
What does overlap protect against?
What goes wrong with very large chunks?