GPUS
224 episodes
#6106: なぜ日本は今も光ディスクを手放さないのか
新幹線を走らせる国が、いまだにブルーレイを買い続ける。最先端技術の国に残る、意外と合理的な理由を探ります。
#6070: The Papers That Moved AI: From AlexNet to This Year's Fights
The classics all said 'learn it from data and scale it up'. This year's most-discussed AI papers are auditing what scaling took for granted.
#6024: What's Actually Inside the KV Cache?
890 bytes per token: why the KV cache is both what makes LLMs practical and what makes long context expensive.
#6013: What Happens Inside One LLM Forward Pass
One trip through the network, layer by layer — and why a classifier needs one pass while an LLM needs hundreds.
#6012: Inside the Decode Loop: How LLMs Generate One Token at a Time
The model doesn't choose tokens or decide when to stop — the engine does. A step-by-step tour of one decode iteration.
#6011: Inside LLM Inference: Prefill, Decode, and the KV Cache
One 8K-token sequence on a 13B model eats 6.7GB of KV cache—while the weights are only 26GB. Here's what actually happens per request.
#6006: Data Residency vs. Inference Residency: What the Law Actually Says
GDPR doesn't require local GPUs. Here's what actually drives geographically restricted AI inference — and the three layers everyone conflates.
#6004: Nobody Can Buy Unlimited AI Inference
Every published tier, every reserved deal, every prepaid reservation — all bounded. Even Azure says it out loud: quota isn't capacity.
#6001: What Do Inference Providers Actually Sell?
Same open weights, 20x cost gap. What Fireworks, Together, and Groq actually sell — and when to just rent a GPU.
#5992: Ridiculous But Legal Ways to Get RAM for AI
Memory prices are up 485% on some kits. Here are ten ridiculous — but completely legal — ways to get RAM for AI without the markup.
#5991: Batch Inference on Your Own GPU: vLLM, SGLang, Ollama, llama.cpp
Hosted batch APIs sell you half-price tokens. Here's how to get the same file-in, file-out workflow on your own Linux box.
#5981: Batch Inference: Four Doors, Half Price
Same tokens, half the price — and almost nobody using it properly. A tour of batch inference from OpenAI to OpenRouter.
#5969: Ubuntu 26.04 LTS: Is Staying on LTS Still a Trade-Off?
Ubuntu 26.04 LTS shipped in April. Point releases, HWE kernels, and SRUs have quietly closed most of the freshness gap — but not everywhere.
#5966: How Much of Your Linux Desktop Is Actually Misconfigured?
Most "Linux tweaks" are folklore. A few are real — and one toggle most people never open is worth up to 42%.
#5964: Containers vs VMs for GPU Inference: What You Actually Pay
Containers cost 0.03%. Passthrough VMs cost 1%... or 36%. Same hardware, different systems — here's how to tell which you'll get.
#5963: Running One Model Across Mismatched GPUs
Tensor, pipeline, or offload? What actually happens when you run one model across several GPUs — including mismatched ones.
#5961: What Your GPU Is Actually Waiting On
Your GPU says 100% utilized. It's multiplying zeros. Here's what the counters actually measure during inference.
#5960: There's No Such Thing as Tokens Per Second
Why one tokens-per-second number can't describe an LLM server — and which metrics actually tell you if it's working.
#5959: What RTF Actually Tells You About Speech Models
Accuracy gets the headlines, but speed decides what ships. How RTF, latency percentiles, and memory shape real ASR deployments.
#5955: Work Lights: Lumens, Lux, and the Specs That Matter
Lumens is the easiest spec to print and the least useful on its own. Here's what to read instead when buying a work light.
#5944: Keeping the Rollout Engine Fed: Inside RLHF Pipelines
Most of an RLHF run is spent waiting on generation, not gradients. Here's how TRL, OpenRLHF, and verl actually differ.
#5911: What Revit 2027 Actually Costs to Run
Michael wants Revit live for AI agents 24/7. The GPU isn't the real bill — the licensing is.
#5907: Rackmounting a Desktop: Size, Cases, and the Swap
How to move a working desktop into a 19-inch rack — sizing by rack unit, picking a chassis that fits your parts, and what the transplant costs.
#5903: Video Tokens, Omni, and the Local VRAM Math
Video costs ~100 tokens per second. That single number explains why ChatGPT hides video upload and why local inference hits a KV cache wall.