BERT
Bidirectional encoder from Google
Episodes
-
#6014: Why Every AI Model Is Secretly a DecoderEncoder, decoder, or both? The three transformer shapes, why one won, and why BERT still refuses to die. -
#5624: One Forward Pass: Building a Bounded ClassifierNo generation, no parsing — just label scores in a single pass. How to build a small encoder classifier that actually works. -
#5621: Constrained Decoding for AI-Generated Podcast TagsA working AI podcast pipeline has one quiet failure: the tagging. Here's how constrained decoding and a stable taxonomy fix it. -
#5438: Why Linguists Left the AI RoomLarge language models grew out of linguistics — so why aren't linguists in the room where they're built? -
#5431: The Other Half of Hugging Face: Why BERT Still Out-Downloads LlamaEncoder models pull over a billion downloads a month. Decoder models pull 397 million. The AI conversation and the download counter are describing ... -
#2639: The Hidden Layer That Makes Search WorkWhy your search results miss the mark — and how cross-encoders fix it. -
#2634: The Two-Stage Pipeline for Persistent User MemoryHow to extract durable personal context from raw prompts and build a self-healing memory layer for AI systems. -
#2622: How Transformers Actually Work: Attention, Tokens, and ContextHow one architectural change unlocked chatbots, image generation, and protein folding — explained without the jargon. -
#2618: Text Normalization's Hidden ComplexityHow to handle acronyms in text-to-speech pipelines using BERT models, lexicons, and layered preprocessing. -
#2487: Why AI Calls Everything a "Prediction" (Even Images)Machine learning calls everything a "prediction" — even generated images. Here's why the terminology matters more than you think. -
#2076: Is Pure NLP Dead? The Hidden Scaffolding of AIModern AI didn't appear from nowhere. Discover how decades of linguistic rules and statistical models built the foundation for today's LLMs. -
#2066: The Transformer Trinity: Why Three Architectures Rule AIWhy did decoder-only models like GPT dominate AI, while encoders and encoder-decoders still hold critical niches? -
#2062: How Transformers Learn Word Order: From Sine Waves to RoPETransformers can’t see word order by default. Here’s how positional encoding fixes that—from sine waves to RoPE and massive context windows. -
#2060: The Tokenizer's Hidden Tax on Non-English TextWhy does a simple greeting in Mandarin cost more to process than in English? It's the tokenizer's hidden inefficiency. -
#1914: Google Invented RAG's Secret SauceBefore LLMs, Google solved the "hallucination" problem with a two-stage trick that's making a huge comeback. -
#1799: The Original AI Blueprints: BERT & CLIPBefore GPT, two models changed everything. Discover how BERT and CLIP taught machines to read and see the world. -
#260: Digital Archeology: The Primitive Power of GPT-1Revisit the 2018 model that started it all. Herman and Corn dive into GPT-1's romance-novel roots and its 117-million-parameter legacy. -
#39: When Smaller AI Is SmarterForget LLMs. Discover SLMs: the specialized, efficient AI powerhouses transforming workflows, from planning to edge devices. -
#13: AI: Not an Overnight Success StoryAI's "overnight success" is a myth. Unravel the true story behind its rise, from humble beginnings to today's innovations. -
#11: How Does Fine Tuning Work Anyway?Unlock the secrets of AI fine-tuning. Discover how your small dataset can shape a giant model.