Transformer Architecture
215 episodes · Page 2 of 9
#6024: What's Actually Inside the KV Cache?
890 bytes per token: why the KV cache is both what makes LLMs practical and what makes long context expensive.
#6021: Why Chat Templates Break Models Silently
Every chat API flattens your messages into special tokens. Get one delimiter wrong and the model just gets quietly dumber.
#6016: What 70 Billion Parameters Actually Means
A single artificial neuron, built from scratch — then the hard part: does that neuron analogy still describe a real transformer?
#6014: Why Every AI Model Is Secretly a Decoder
Encoder, decoder, or both? The three transformer shapes, why one won, and why BERT still refuses to die.
#6013: What Happens Inside One LLM Forward Pass
One trip through the network, layer by layer — and why a classifier needs one pass while an LLM needs hundreds.
#6012: Inside the Decode Loop: How LLMs Generate One Token at a Time
The model doesn't choose tokens or decide when to stop — the engine does. A step-by-step tour of one decode iteration.
#6011: Inside LLM Inference: Prefill, Decode, and the KV Cache
One 8K-token sequence on a 13B model eats 6.7GB of KV cache—while the weights are only 26GB. Here's what actually happens per request.
#5975: Merging vs Distillation: Can Two Models Become One?
Merging and distillation get lumped together, but they're opposites. One composes existing capability; the other has to teach it.
#5974: One Model or Two? ASR Cleanup Explained
If speech models can punctuate and capitalize on their own, why does a separate 484MB cleanup model exist at all?
#5945: How a Reward Model Actually Trains an LLM
One number, no gradients, four models in the room. What a reward model actually does inside the RLHF loop.
#5899: Same GPU, Same Weights, Different Answers
Identical hardware, identical weights, different engine — and answers that shift by up to 16.6 points.
#5796: 900 Tonnes on Public Roads: The Weirdest Cargo Moves
544 wheels, 66 hours on the road with three elephants, and a spaceship on Crenshaw Boulevard — the strange industry that moves the impossible.
#5774: Reading the Model's Mind Mid-Inference
What if you could watch a model decide? Inside the tools that open up inference mid-computation — and the limits they hit.
#5626: What Multimodal Models Actually Learn From Video
Late fusion, early fusion, and vision laziness — what actually happens when models train on video, audio, and text together.
#5625: Omni and the Marble That Rolls Uphill
Google's Omni claims an intuitive grasp of gravity and momentum. Can a video model really learn physics, or just remember what physics looks like?
#5624: One Forward Pass: Building a Bounded Classifier
No generation, no parsing — just label scores in a single pass. How to build a small encoder classifier that actually works.
#5583: The Last Lighthouse Keepers: Life at the Edge of the Sea
Before automation, someone had to trim the wick, crank the clockwork, and keep the light burning through the storm. What was that job really like?
#5567: The Sinai Crossing Where a Stamp Draws the Border
A 24-hour pedestrian crossing between Eilat and Taba, where a free stamp, a one-kilometer fee waiver, and four treaty zones shape every trip into S...
#5561: Fog, Light, and the Limits of Sight
Fog is a cloud sitting on the road. Here's the physics of why it forms fast, what it does to light, and why there's no safe driving visibility.
#5520: Keeping a Fine-Tune Alive Across Model Releases
Daniel wants to fine-tune DeepSeek Flash 4.1 on edited podcast scripts. The hard part isn't training — it's surviving the next release.
#5447: The Models That Never Talk Back
Some models read text, score it, and return a number. No chat, no reasoning, just decisions — and they're running under every router you use.
#5443: Heads vs Layers: How Model Merging Actually Works
Heads aren't the Lego bricks of model merging — layers are. Here's what heads really do and how frankenmerges get built.
#5442: When Do LLMs Suddenly Get Good at Things?
New capabilities seem to appear out of nowhere as models scale. Whether that's real — or just how we measure — is still an open fight.
#5438: Why Linguists Left the AI Room
Large language models grew out of linguistics — so why aren't linguists in the room where they're built?