Ch 6 Unsupervised Training (TSDAE)Apply

Building sentence embedders when you have text but no labels — the common real-world situation.

Core concepts

  • No labels, still learn. Unsupervised methods learn embeddings from raw text alone.
  • TSDAE (denoising auto-encoder). Corrupt a sentence (delete/swap words), encode it into a single vector, then force a decoder to reconstruct the original — the bottleneck vector must capture meaning.
  • Use as a starting point. Unsupervised pre-training followed by light supervised fine-tuning is a powerful, data-efficient combo.

What you must master

  • Explain when unsupervised training is the right choice (no labels available) Level 1
  • Describe the TSDAE corrupt–encode–reconstruct loop Level 1
  • Run unsupervised pre-training on a raw domain corpus Level 2
  • Combine unsupervised pre-training with later supervised fine-tuning Level 2

Architect’s lens

Most enterprises have plentiful unlabelled text and almost no labelled pairs. This chapter unlocks the “cold start” strategy: bootstrap a domain embedder from raw documents, then refine. It sets up Part 4’s more advanced domain-adaptation techniques.