Ch 4 Multiple Negatives Ranking (MNR) LossApply

The modern default for training high-quality retrieval embedders.

Core concepts

  • Contrastive learning. Pull a query and its positive (relevant) pair together; push apart negatives. This directly optimizes for retrieval ranking.
  • In-batch negatives. For each positive pair in a batch, all other examples act as negatives — free, plentiful negatives without extra labelling.
  • Data format. Needs only (anchor, positive) pairs — e.g. (question, answer) or (query, relevant passage). Easier to source than labelled triplets.
  • Batch size matters. Larger batches = more negatives = better embeddings.
Anchors(queries)Positives(relevant passages)A1A2A3P1P2P3pull togetherpush apart(in-batchnegatives)similarity matrix (batch size 3)1.00.10.20.01.00.10.00.21.0
MNR loss: each anchor is pulled toward its own positive (diagonal) and pushed away from every other pair’s positive in the batch (off-diagonal, free negatives).

Under the hood: that similarity matrix’s diagonal is the target for PyTorch’s CrossEntropyLoss — see Chapter 3.5 for a goal/input/target/code comparison of CrossEntropyLoss in MNR loss vs. Softmax loss.

What you must master

  • Explain contrastive learning and the role of in-batch negatives Level 1
  • Prepare (anchor, positive) pair data for MNR training Level 2
  • Fine-tune an embedder with MNR loss and tune batch size Level 2
  • Explain why larger batches and hard negatives improve retrieval quality Level 2

Architect’s lens

MNR is the objective behind most strong retrieval models — and the one you’ll most often use to fine-tune an embedder to a client’s domain using nothing more than positive pairs mined from logs, FAQs, or docs. Understanding in-batch negatives explains why training infra (GPU memory for big batches) directly affects retrieval accuracy — a real cost/quality trade-off you’ll size.