Training Data
322 episodes · Page 3 of 14
#6217: Fine-Tuning a Trillion-Parameter Model at Home
What does it actually take to full fine-tune a trillion-parameter model? Sixteen terabytes of optimizer state, hundreds of GPUs, and a very big bill.
#6214: Nobody Trains an Uncensored Model From Scratch
Every "uncensored" model is a renovation, not a new build — and the political lean lives in the agreement layer, not the refusals.
#6213: Ten Weirdest AI Evals Ever Published
A pelican on a bicycle, a badger for Groundhog Day, and a benchmark where passing is the bad outcome — the strangest AI evals ever run.
#6212: Woolly Sedate Snail and Other Model Failures
Most of Hugging Face's 2.96M models sit unread. But the dust has a personality — snails, pirates, and a matchbox that resigns.
#6196: Fine-Tuning ASR: When Your Model Memorizes You
Your fine-tuned speech model aces your training recordings — but does it actually work on fresh audio?
#6194: Teaching a TTS Model His Own Accent
One man fine-tuned a text-to-speech model on his own voice to get Irish-accented English that can say Hebrew words mid-sentence.
#6192: What Actually Happens When You Train a Model
Data in, model out — except the "data" is pairs, the split is the whole game, and one leaked speaker can fake your score.
#6191: What Actually Happens During Fine-Tuning
Sixteen bytes per parameter in, two out — a walk through the forward pass, backward pass, and optimizer that turn a dataset into new weights.
#6189: What "Real" Improvement Actually Means in AI Evals
A p-value isn't the probability your result was luck. Here's what chance actually means in AI benchmarks — and why bold numbers need cross-examinat...
#6185: Why AI Labs Sell a Tier Above the Flagship
Fable, Deep Think, SuperGrok Heavy — what the $300/month top tier actually does that the flagship can't.
#6184: Anthropic's Unreleased Model Two, Explained
Anthropic trained a model better than what it sells — and never released it. What that gap actually tells us.
#6180: Mapping Code-Switching: 113 Pairs, No Community
Researchers count 113 code-switched language pairs — but a pair is just a label on a file, not a community of speakers.
#6178: Why Your Voice Clone Sounds American
Your cloned voice drifts toward American English. Here's what's actually happening — and why "neutral" is a myth.
#6177: How Voice Cloning Works From Six Seconds of Audio
Six seconds of reference audio is all Chatterbox needs. The real work happened in the half-million hours of training before you showed up.
#6175: Video Models Can't Tell Time: What Motion Benchmarks Reveal
Frontier models score 48% on tasks where humans hit 98% — the gap is motion, speed, and sequence, not scene description.
#6174: What Your Voice Reveals: Accent, Distance, and Movement in Audio AI
What can AI actually hear in your voice? Accent detection, speaker distance, whale songs, and the gap between deployed audio models and chatty gene...
#6173: Auditing a Sycophantic Model From the Inside
A stumped lab, a fawning model, and the diagnostic playbook nobody has a name for — from eval slicing to persona vectors.
#6171: AI Audio Restoration: What Actually Ships in 2026
Noise removal and bandwidth extension are shipping products today. Matching a clip to a target EQ is still mostly a research prototype.
#6168: Hebrish: Hebrew-Aware Dictation That Actually Works
Two new speech models fix English-Hebrew code-switching dictation — and the whole training run cost about fifty cents of GPU time.
#6161: Turning Words Into Sound: G2P, IPA, and the Hebrew Vowel Problem
Text-to-phonetics tools exist, but the neural ones guess wrong on exactly the words you care about — names, brands, and Hebrew.
#6159: When a Speech Model Can't Decide How to Write Dollars
One dataset says "$20," another says "twenty dollars." Fine-tuning a speech model means finding out which convention wins — and why it matters.
#6158: Naming the ASR Sub-Tasks Nobody Mapped
A 2001 linguistics paper already named Daniel's problem — but it points the other way. So what do you call a fine-tune set that expands "F Y I" to ...
#6154: Who Checks the Labels? Weak vs Strong Supervision
Human labels, rule-based labels, or a teacher model writing the exam — what actually makes supervision "strong," and who verifies it?
#6148: The Tiny Hebrew Models Behind the Show's Voice
Dicta's 300M-parameter nikudizer, Phonikud's adaptors, and the ktiv male model hiding inside a vowel adder.