Ch 13 Generative Pseudo-Labeling — GPLApply

The state-of-the-art unsupervised domain-adaptation pipeline — GenQ upgraded.

Core concepts

  • GPL = GenQ + hard negatives + soft labels. A three-stage pipeline for domain adaptation with zero labels.
  • 1 · Query generation. Generate queries per passage (as in GenQ).
  • 2 · Negative mining. Retrieve hard negatives for each generated query.
  • 3 · Pseudo-labeling. A cross-encoder scores each (query, passage) pair to produce a soft relevance margin; the bi-encoder is trained with MarginMSE to match those scores.
  • Why it wins. Soft margins are more informative and noise-tolerant than binary positive/negative labels.
1. Generate queriesT5 per passage2. Mine hard negativesretrieve near-misses3. Pseudo-labelcross-encoder marginsMarginMSEtrain bi-encoder
GPL’s three stages, ending in MarginMSE training on soft cross-encoder scores.

What you must master

  • Explain GPL’s three stages and how it improves on GenQ Level 1
  • Explain soft labels / MarginMSE vs. binary labels Level 2
  • Run GPL to domain-adapt a retriever with no labelled data Level 2
  • Choose GPL vs. GenQ vs. simple fine-tuning for a project’s constraints Level 3

Architect’s lens

GPL is your best default answer to “how do we get a great retriever for this niche corpus with no labels?”. Knowing its cost (query generation + negative mining + cross-encoder scoring over the corpus) lets you budget the adaptation pipeline and decide whether the quality lift justifies the compute — the essence of a solution architect’s job.