Ch 2 Sentence Transformers & EmbeddingsDesign

How to get one high-quality vector per sentence — and why naive BERT pooling fails.

Core concepts

  • The BERT problem. Vanilla BERT produces per-token vectors; comparing sentences by feeding pairs through BERT (cross-encoding) is accurate but O(n²) and far too slow for search over millions of items.
  • Sentence-BERT (SBERT). A bi-encoder: each sentence is embedded independently into one vector, so you embed once and reuse. Pooling (mean/CLS) collapses token vectors into a sentence vector.
  • Bi-encoder vs. cross-encoder. Bi-encoders are fast and scalable (good for retrieval); cross-encoders are more accurate (good for re-ranking a short list).
  • Semantic similarity applications. Search, clustering, paraphrase mining, deduplication, and semantic textual similarity (STS).
Bi-encoder (retrieval)Sentence ASentence Bencoderencodervec Avec Bcosine(vecA, vecB)Cross-encoder (re-rank)[A + B] togetherencoderrelevance score
Bi-encoders scale to millions (embed once); cross-encoders re-score a small candidate set for precision.

What you must master

  • Explain why SBERT exists and why cross-encoding doesn’t scale to search Level 1
  • Produce sentence embeddings with sentence-transformers and run a similarity search Level 2
  • Describe pooling strategies (mean vs CLS) and their effect Level 2
  • Design a two-stage retrieve-then-rerank pipeline (bi-encoder → cross-encoder) Level 3

Architect’s lens

The bi-encoder + cross-encoder pattern is the backbone of production retrieval and RAG. You decide the split: fast bi-encoder recall over the whole corpus, then an accurate cross-encoder to re-rank the top-K. This is your primary lever to trade latency and cost against answer quality.