Retrieval-Augmented Generation: The Pipeline, the Evidence and the Paper Map
Published:
Two memories, several decisions
A language model stores statistical information in its parameters. A retrieval system exposes an external collection that can be searched and updated separately. The original RAG paper combines a learned retriever and generator; modern engineering also uses the term for retrieve–prompt–generate pipelines that do not implement its latent-document objective.
That distinction matters. Replacing documents in a prompt is different from training a model to marginalise over retrieved passages. Both can use external evidence, but their training signals, interfaces and failure modes differ.
A minimal pipeline contains ingestion, indexing, query processing, candidate retrieval, optional reranking, context construction, generation and evidence checks. Changing any stage changes the behaviour of the whole system.
Follow one question end to end
Suppose a user asks whether the R7 robot supports outdoor charging. The corpus contains an old R7 manual, a current R7 safety note and an R8 brochure. A semantic search can return all three because their vocabulary is similar.
A good pipeline preserves the product identifier and version constraint, prefers the applicable R7 evidence, and keeps the qualification attached to the charging statement. The generator must not borrow the R8 feature or average the old and new instructions into a compromise. If the documents genuinely conflict, the answer should explain the conflict with the relevant dates and sources.
A fluent answer can therefore fail despite successful similarity search. Relevance is a relation to a question; applicability includes scope and version; support is a relation between a source and a specific claim.
A probability model and an engineering model
A latent-document formulation writes
The retriever distributes mass over documents; the generator predicts an answer conditional on one. Practical systems usually approximate the sum or use a selected set as a combined context. Concatenating documents into one prompt is not automatically the same computation as marginalising their conditional answer probabilities.
For diagnosis, use a conditional success chain. If sufficient evidence is retrieved with probability 0.9, retained in context with probability 0.8 given retrieval, and used correctly with probability 0.85 given both, the probability of that successful path is \(0.9\times0.8\times0.85=0.612\). These are conditional rates, so the product does not assume independence. Other paths, including correct answers from parametric knowledge, should be measured separately.
Before generative RAG: DrQA and ORQA
DrQA couples document retrieval with an extractive reader over Wikipedia. Its answer is a passage span, which makes it an important precursor to the retrieval–reader decomposition. ORQA learns a dense retriever with latent evidence using question–answer supervision; supporting documents need not be provided as training labels. Together they motivate two questions that recur throughout this book: how does the reader consume evidence, and what supervision teaches the retriever to find it?
RAG extends this lineage with a generative answer distribution. A generated sentence can combine information and produce wording absent from the source, so span extraction metrics alone no longer establish evidence support.
The research families
| Question | Chapters and representative papers |
|---|---|
| How do we find evidence? | DPR/Contriever, ColBERT/SPLADE, hybrid reranking |
| How does retrieval enter learning? | REALM, FiD/Atlas, RETRO, kNN-LM, REPLUG |
| When should we search again? | Self-RAG, CRAG/Adaptive-RAG, FLARE |
| How do we answer multi-step questions? | IRCoT/ReAct, Search-R1 |
| How do we organise corpus-wide evidence? | RAPTOR, GraphRAG, LightRAG |
| How do we know the answer is supported? | RAGAS/ARES, ALCE and attribution, long-context analysis |
These are connected reading routes through influential methods, not interchangeable product labels. The chapters explain both the papers’ contributions and worked examples of their implications.
Establish a baseline before adding components
Start with a versioned corpus, a lexical retrieval baseline, a fixed context budget and a question set with supporting passages. Record retrieved IDs, scores, selected context and output claims. Add one change at a time: dense retrieval, query expansion, a reranker or an adaptive loop.
Measure evidence recall, answer support and cost together. More retrieval calls or a longer prompt can improve one metric while increasing delay or distracting the generator. Oracle-evidence experiments help distinguish a retrieval bottleneck from a generator that cannot use the right evidence even when it is supplied.
Treat retrieved text as source material, with its own authority and uncertainty. Content inside a document does not become an instruction merely because the retriever returned it. For systems with document permissions, apply the authorised corpus boundary before information reaches the generator.
Recap
- Separate parametric memory from the searchable corpus.
- Distinguish the original probabilistic RAG model from prompt-based RAG pipelines.
- Measure retrieval, context retention, generation and attribution separately.
- Use a fixed corpus and budget when comparing paper-inspired changes.
References
- Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS, 2020.
- Karpukhin et al. Dense Passage Retrieval for Open-Domain Question Answering. EMNLP, 2020.
- Gao et al. Enabling Large Language Models to Generate Text with Citations. EMNLP, 2023.
- Chen et al. Reading Wikipedia to Answer Open-Domain Questions. ACL, 2017.
- Lee, Chang and Toutanova. Latent Retrieval for Weakly Supervised Open Domain Question Answering. ACL, 2019.