Transformer Architecture
215 episodes
#6384: Why LLM Output Tokens Cost More Than Input
Decode computes at just 1.4% utilization on an H100 because reading 140 gigabytes of weights takes 280 times longer than the arithmetic.
#6379: Porting Voice Models to Core ML and MLX
Core ML packages separate computation specs from raw weights, while MLX runs on CPU and GPU without touching the Neural Engine.
#6368: What Does It Take To Train An AI Model From Scratch?
nanochat trains a BPE tokenizer in Rust in roughly 34 seconds, yet it can quietly disadvantage languages like Korean by nearly half.
#6345: How vLLM Squeezes More Requests into GPU Memory
Early serving engines wasted up to 80% of KV cache memory on unspent reservations; vLLM cut that slack to under 4% using virtual memory paging.
#6311: How Transformers Process Images and Video
A Vision Transformer divides a standard image into a grid of 16-by-16 pixel patches rather than processing each individual pixel as a token.
#6288: What Happens Between Model Checkpoints
Extending reinforcement learning on Olmo 3 Think for 21 days yielded a 20-point leap on IFBench without starting fresh pretraining.
#6273: How Timestral Forecasts Time Series Data
Mistral's open-weight Timestral models predict up to 1,024 future steps zero-shot using grouped patches across twenty-one non-crossing quantiles.
#6253: Why Context Windows Stalled at a Million Tokens
A dense 70B model requires about 328 gigabytes of KV cache to serve a single one-million-token prompt, exceeding the memory of an Nvidia B300 chip.
#6245: Why Inference Engines Change Benchmark Scores
A study found a 16.6 percentage point spread on GSM8K across different inference engines running identical weights on the same GPU.
#6243: Inside the Mixture of Experts Router
What actually decides which expert handles your token? A look inside the router, from sigmoid affinities to load balancing.
#6220: Mixture of Experts: Bigger Than It Looks
MoE models store a trillion parameters but only fire a few billion per token. Here's what the active number really measures — and why smaller isn't...
#6211: Attention Is Still Winning: The Hybrid Model Census
Mamba layers shipped — inside hybrids. A census of Qwen, Nemotron, Kimi, Granite, and why MiniMax went back to full attention.
#6191: What Actually Happens During Fine-Tuning
Sixteen bytes per parameter in, two out — a walk through the forward pass, backward pass, and optimizer that turn a dataset into new weights.
#6178: Why Your Voice Clone Sounds American
Your cloned voice drifts toward American English. Here's what's actually happening — and why "neutral" is a myth.
#6174: What Your Voice Reveals: Accent, Distance, and Movement in Audio AI
What can AI actually hear in your voice? Accent detection, speaker distance, whale songs, and the gap between deployed audio models and chatty gene...
#6169: What It's Like to Be a Token Inside an LLM
A token's-eye view of an LLM's forward pass — no cone, no squeezing, just a hallway where every layer scribbles on you.
#6148: The Tiny Hebrew Models Behind the Show's Voice
Dicta's 300M-parameter nikudizer, Phonikud's adaptors, and the ktiv male model hiding inside a vowel adder.
#6110: The Chatbot OpenAI Almost Shelved
Instruction models existed for months. Then a rumour about Anthropic pushed OpenAI to ship a chat box in two weeks.
#6105: Не только Китай: ИИ в России, Индии и Японии
Китай занимает почти всё внимание в разговорах об ИИ, но Россия, Индия и Япония тоже создают мощные региональные модели с необычной архитектурой.
#6072: China Beyond Beijing and Shanghai: A Regional Guide
Most Westerners know two Chinese cities. Here's the rest of the map — regions, megacities you've never heard of, and which ones you can actually re...
#6070: The Papers That Moved AI: From AlexNet to This Year's Fights
The classics all said 'learn it from data and scale it up'. This year's most-discussed AI papers are auditing what scaling took for granted.
#6052: What's Actually in the Hugging Face ASR Catalog?
Four real architectural bets sit underneath thousands of Hugging Face ASR repos. Here's how to tell the bets from the repackagings.
#6029: One Model, Two Modes: Does AI Consolidation Hold Up?
OpenAI hides a router, DeepSeek merged two modes, Anthropic removed the knob. Is the model landscape really consolidating?
#6027: Can You Snap Two Fine-Tunes Together Like LEGO?
Two fine-tunes, one base model, and a question from Jerusalem: can you merge Hebrew code-switching with technical vocab — without retraining?