Chapter 11 · Retrieval and memory2020
Retrieval-augmented generation (RAG)
RAG fetches relevant documents first and puts them in the prompt, so answers are grounded in real sources.
Chapter 11 · Retrieval and memory2020
RAG fetches relevant documents first and puts them in the prompt, so answers are grounded in real sources.
A question arrives. The model does not know your documents.
RAG adds one step before the answer: fetch the relevant cards, then write.
It has never seen your school rules, your company files or this morning's news. Ask about them and it can only guess.
Pick a question. The matched words are highlighted in the retrieved notes. Then turn retrieval off, or ask the question that is not in the handbook.
Use only these notes to answer. [1] The library opens at 9 am and closes at 6 pm from Monday to Saturday. It is shut on Sundays. [2] To join the library, bring a photo and your school ID card to the front desk. Membership is free for students. Question: When does the library close?
A teaching version: the search is a simple word match and the answer is scripted from the best note. Real RAG searches by meaning and a real model writes the reply. Notes that match only a little can still be fetched, which is why too many notes can add noise.
What does RAG add before the model writes its answer?
Why can RAG answer about your private files?
Nothing relevant is found. What should a good RAG system do?