Ch 8 Retrievers for Question-AnsweringDesign

Fine-tuning the model that decides which passages the LLM ever sees.

Core concepts

  • Dense Passage Retrieval (DPR). Two encoders — one for questions, one for passages — trained so a question sits close to its answer passage.
  • Asymmetric search. Queries are short; passages are long. Retrievers are trained to bridge this length/style gap (unlike symmetric sentence-similarity).
  • Hard negatives. Training with plausible-but-wrong passages (e.g. BM25 near-misses) sharpens the retriever far more than random negatives.
  • Evaluation. Recall@K / top-K accuracy — does the correct passage appear in the retrieved set?

What you must master

  • Explain DPR’s dual-encoder design and asymmetric (query≠passage) retrieval Level 1
  • Fine-tune a retriever and mine hard negatives Level 2
  • Measure retrieval with Recall@K / top-K accuracy Level 2
  • Decide between an off-the-shelf vs. fine-tuned retriever for a domain Level 3

Architect’s lens

The retriever is the highest-leverage component in a RAG system — it caps the ceiling on answer quality. Your job is to decide when a generic embedding API suffices vs. when a fine-tuned domain retriever with hard negatives is worth the MLOps cost, and to define the Recall@K SLO the whole system is judged against.