Fine Tuning
178 episodes · Page 2 of 8
#6244: When Your Fine-Tune Isn't Done Learning Yet
Falling loss, 90% word error rate, and a model that still says "Derry gym lag." What undertraining actually looks like — and when more epochs won't...
#6242: Training a Model on Your Own Taste
What does it actually take to train a small model on your editorial taste using pairwise preferences and DPO?
#6241: Why Your Synthetic Data Isn't as Diverse as You Think
Fifty-nine percent of one GPT-4 dataset started with "Once upon a time." Correct examples can still make a terrible training set.
#6240: Gold vs Silver Training Data: What Makes Labels Trustworthy
Human-checked "gold" labels and machine-made "silver" ones aren't as different as the names suggest — and the gap matters.
#6239: Testing Synthetic Data Generators Before You Scale
A benchmark trains student models on each generator's output to find out which model actually writes useful training data.
#6238: Which AI Models Actually Generate Good Training Data?
Open weights vs closed APIs for synthetic data, and why IFEval scores don't tell you which model will hold up.
#6237: Small Models, Big Questions: When Cheap Fables Get Expensive
Three million fables for pennies — but can a small model tell a good example from a plausible one?
#6223: How Many Epochs Does Fine-Tuning Actually Need?
Epochs are a receipt, not a plan. How to pick a training ceiling, read the curves, and know when to stop.
#6222: Why Fine-Tuning Runs Can't Always Resume
Eighty cents and twenty-five minutes per training run — so why does the checkpoint question still keep a tinkerer up at night?
#6221: How Many Examples Does Fine-Tuning Actually Need?
Everyone says "thousands." Nobody says how many for your task. Here's how to measure the number yourself.
#6220: Mixture of Experts: Bigger Than It Looks
MoE models store a trillion parameters but only fire a few billion per token. Here's what the active number really measures — and why smaller isn't...
#6218: Training a LoRA to Write Like You
A Humanizer model stripped every trace of the author. The fix isn't a better Humanizer — it's a LoRA trained on your own writing.
#6217: Fine-Tuning a Trillion-Parameter Model at Home
What does it actually take to full fine-tune a trillion-parameter model? Sixteen terabytes of optimizer state, hundreds of GPUs, and a very big bill.
#6214: Nobody Trains an Uncensored Model From Scratch
Every "uncensored" model is a renovation, not a new build — and the political lean lives in the agreement layer, not the refusals.
#6196: Fine-Tuning ASR: When Your Model Memorizes You
Your fine-tuned speech model aces your training recordings — but does it actually work on fresh audio?
#6192: What Actually Happens When You Train a Model
Data in, model out — except the "data" is pairs, the split is the whole game, and one leaked speaker can fake your score.
#6191: What Actually Happens During Fine-Tuning
Sixteen bytes per parameter in, two out — a walk through the forward pass, backward pass, and optimizer that turn a dataset into new weights.
#6160: Where Do Tiny Speech Rules Live? A Ladder From Regex to LoRA
A regex notebook, a WFST grammar, or a fine-tune? Where do tiny speech-rewrite rules actually belong in a 600M-parameter model?
#6159: When a Speech Model Can't Decide How to Write Dollars
One dataset says "$20," another says "twenty dollars." Fine-tuning a speech model means finding out which convention wins — and why it matters.
#6157: Where to Sit When You're Fine-Tuning Your Own Voice
Your phone's built-in mic is the point — the setup around it is what ruins the takes. Rooms, remotes, and the app setting that deletes your data.
#6154: Who Checks the Labels? Weak vs Strong Supervision
Human labels, rule-based labels, or a teacher model writing the exam — what actually makes supervision "strong," and who verifies it?
#6092: 到底是谁在塑造中国的人工智能思想？
不只是实验室的老板：那些定义中国如何打造、又如何争论人工智能的研究者和思想。
#6078: Reading Polymarket's Israel Election Prices
25 live markets, $47M on one question — what the money says about Israel's next prime minister.
#6055: How to Fine-Tune ASR on Noisy Audio
A small noise library mixed into clean speech beats collecting hours of noisy recordings. Here's the ratio that actually works.