Training Data
322 episodes · Page 2 of 14
#6316: Why Protein Design Ignores Astronomical Counts
A hundred amino acids can form roughly 10 to the 130th possible sequences, far exceeding the number of atoms in the observable universe.
#6315: Inside China's Mixed-Language AI Models
Domestic Chinese models typically draw sixty to eighty percent of their training data from Chinese, leaving massive room for English and code.
#6311: How Transformers Process Images and Video
A Vision Transformer divides a standard image into a grid of 16-by-16 pixel patches rather than processing each individual pixel as a token.
#6306: Training Custom Rerankers for Podcast Search
Expanding a retrieval candidate pool past 300 items yielded less than a single percentage point of recall in legal benchmark evaluations.
#6300: Choosing the Right Audio Annotation Tool
Desktop tool ELAN supports independent timeline tiers to prevent overlapping sounds like car horns from flattening into speech transcripts.
#6299: Finding Recurring Urban Sounds With Four Examples
In DCASE evaluations, prototypical network baselines can locate specific sound events across long recordings using as few as five labeled samples.
#6285: Why Coding Benchmarks Split Across Models
GLM 5.3 claims 88.2 on Terminal Bench 2.1 against Opus 4.8 at 85, but newer evaluations reveal a very different order.
#6274: Machine Learning for Smells, Whale Codas, and Aftershocks
A graph neural network mapped molecular structures to scents, matching a consensus panel better than the typical individual human nose.
#6272: Teaching Audio Models to Hear Beyond Words
Speech recognition rewards ignoring background noise, but audio-language models train specifically on environmental sounds and temporal scenes.
#6266: Balancing Speech Training Data With Square Roots
A fine-tuning recording app weighs tasks by the square root of unread sentences so massive datasets don't crowd out smaller prompt sets.
#6265: Why Upgrading Embedding Models Demands Reindexing
Cohere's Embed Five Fast queries against a Pro index reach 98.4 percent of performance without regenerating stored document vectors.
#6264: Choosing Omni Embedding Models for Mixed Media
Jina's omni model trains only 0.35 percent of its weights while aligning with existing text indices.
#6263: Why Numeric Data Lacks a Default Embedding Model
Popular time-series models like IBM's Granite TSPulse forecast trends rather than provide canonical vectors for similarity search.
#6244: When Your Fine-Tune Isn't Done Learning Yet
Falling loss, 90% word error rate, and a model that still says "Derry gym lag." What undertraining actually looks like — and when more epochs won't...
#6242: Training a Model on Your Own Taste
What does it actually take to train a small model on your editorial taste using pairwise preferences and DPO?
#6241: Why Your Synthetic Data Isn't as Diverse as You Think
Fifty-nine percent of one GPT-4 dataset started with "Once upon a time." Correct examples can still make a terrible training set.
#6240: Gold vs Silver Training Data: What Makes Labels Trustworthy
Human-checked "gold" labels and machine-made "silver" ones aren't as different as the names suggest — and the gap matters.
#6239: Testing Synthetic Data Generators Before You Scale
A benchmark trains student models on each generator's output to find out which model actually writes useful training data.
#6238: Which AI Models Actually Generate Good Training Data?
Open weights vs closed APIs for synthetic data, and why IFEval scores don't tell you which model will hold up.
#6237: Small Models, Big Questions: When Cheap Fables Get Expensive
Three million fables for pennies — but can a small model tell a good example from a plausible one?
#6236: Synthetic Logs Without Leaking Your Clients
You can rewrite your logs with a cheap model — but can you keep the task boundaries while losing the client names?
#6223: How Many Epochs Does Fine-Tuning Actually Need?
Epochs are a receipt, not a plan. How to pick a training ceiling, read the curves, and know when to stop.
#6222: Why Fine-Tuning Runs Can't Always Resume
Eighty cents and twenty-five minutes per training run — so why does the checkpoint question still keep a tinkerer up at night?
#6221: How Many Examples Does Fine-Tuning Actually Need?
Everyone says "thousands." Nobody says how many for your task. Here's how to measure the number yourself.