Series

Small Models and Distillation

Small language models, classifiers, distillation and model merging as cheaper alternatives to frontier LLMs.

23 episodes
  1. 1

    When Smaller AI Is Smarter

    Forget LLMs. Discover SLMs: the specialized, efficient AI powerhouses transforming workflows, from planning to edge devices.

    22:40 · 9 Dec 2025
  2. 2

    Why Tiny Digital Savants Are Outperforming God-Models

    Are massive AI models hitting a wall? Discover why the future belongs to lean, domain-specific "digital savants" and vertical pre-training.

    31:45 · 26 Feb 2026
  3. 3

    The AI Long Tail: How Small Models Outsmart the Giants

    Discover why 31B models are outperforming GPT-5.4 in reasoning and how the AI "long tail" provides the key to local sovereignty and accuracy.

    21:57 · 24 Mar 2026
  4. 4

    Dark Knowledge: The Art of AI Model Distillation

    Discover how model distillation transfers "dark knowledge" from massive AI giants into tiny, efficient models that live in your pocket.

    20:44 · 26 Mar 2026
  5. 5

    Microsoft's Phi: The Small Model Bet for Agentic AI

    Microsoft is pushing small language models like Phi for agentic AI. Here’s why that strategy matters for speed, cost, and edge computing.

    26:45 · 29 Mar 2026
  6. 6

    Where Intelligence Should Live in Your Pipeline

    When should you fine-tune a tiny model for prompt enhancement instead of prompting a large one? The answer depends on latency, precision, and domain.

    33:24 · 27 Apr 2026
  7. 7

    Model Identity Fraud: Distillation or Data Contamination?

    When AI models lie about who they are, is it stolen knowledge or just contaminated training data?

    31:31 · 8 Sept 2026
  8. 8

    Chaining Small Models for Dictation Cleanup

    Daniel's Android dictation fork won't render "three point five" as a decimal. How many models does cleanup actually need?

    31:24 · 20 Sept 2026
  9. 9

    Teaching a Small Model to Stop Spelling Out Numbers

    Your ASR pipeline is fine until someone dictates "three point two" and gets "three point two" spelled out. Here's how inverse text normalization ac...

    22:31 · 20 Sept 2026
  10. 10

    Chaining Small Models for Voice Cleanup

    Six cleanup stages at 97% accuracy each compound to 83% end-to-end. So how many small models can you actually chain?

    22:34 · 20 Sept 2026
  11. 11

    The Other Half of Hugging Face: Why BERT Still Out-Downloads Llama

    Encoder models pull over a billion downloads a month. Decoder models pull 397 million. The AI conversation and the download counter are describing ...

    22:36 · 21 Sept 2026
  12. 12

    Small Models as Rewriters, Not Writers

    Why "don't say X" prompts backfire, and how a tiny grammar-constrained model can scrub a script without breaking its grammar.

    19:59 · 21 Sept 2026
  13. 13

    When Small NLP Models Beat the LLM

    Feature extraction, fill-mask, token classification — the classic NLP tasks still have a job. Here's when a small model beats a frontier API.

    24:27 · 21 Sept 2026
  14. 14

    Heads vs Layers: How Model Merging Actually Works

    Heads aren't the Lego bricks of model merging — layers are. Here's what heads really do and how frankenmerges get built.

    22:46 · 22 Sept 2026
  15. 15

    Small Models, Big Schemas: When JSON Constraints Backfire

    Small models plus strict JSON schemas should be a safe bet. A 15,000-generation study found the opposite.

    20:21 · 22 Sept 2026
  16. 16

    Small Models, Big Guardrails: PII Detection in ChatGPT

    A tiny 600M parameter model is quietly scanning your ChatGPT tool calls for PII. Here's how that class of guardian model actually works.

    25:54 · 23 Sept 2026
  17. 17

    One Forward Pass: Building a Bounded Classifier

    No generation, no parsing — just label scores in a single pass. How to build a small encoder classifier that actually works.

    21:52 · 28 Sept 2026
  18. 18

    Freeze the Model, Move the Labels

    Your classifier isn't stale — your label set moved. Why retraining is cheap, and why not retraining might be smarter.

    22:31 · 9 Oct 2026
  19. 19

    Text Classifiers: Local Models vs Always-On Endpoints

    A specialist classifier costs $5.70 per million messages. A general LLM costs $142. So why is the obvious product so hard to find?

    26:22 · 9 Oct 2026
  20. 20

    Training a Model When You Don't Know the Labels Yet

    Daniel wants to extract adapter specs from his shelf — but he can't name the labels until he's already annotating.

    26:33 · 13 Oct 2026
  21. 21

    Merging vs Distillation: Can Two Models Become One?

    Merging and distillation get lumped together, but they're opposites. One composes existing capability; the other has to teach it.

    24:40 · 14 Oct 2026
  22. 22

    Hebrew in Latin Letters: A TTS Pipeline Fix

    A token classifier that finds Hebrew words written in Latin letters so TTS stops mangling them — built for under $3.

    33:28 · 15 Oct 2026
  23. 23

    Small Models, Big Questions: When Cheap Fables Get Expensive

    Three million fables for pennies — but can a small model tell a good example from a plausible one?

    21:43 · 24 Oct 2026