AI Inference
549 episodes · Page 4 of 23
#6185: Why AI Labs Sell a Tier Above the Flagship
Fable, Deep Think, SuperGrok Heavy — what the $300/month top tier actually does that the flagship can't.
#6184: Anthropic's Unreleased Model Two, Explained
Anthropic trained a model better than what it sells — and never released it. What that gap actually tells us.
#6173: Auditing a Sycophantic Model From the Inside
A stumped lab, a fawning model, and the diagnostic playbook nobody has a name for — from eval slicing to persona vectors.
#6170: One Endpoint, Many Boxes: Local LLM Routing
A LiteLLM-style proxy can fail over between your own GPU boxes and the cloud — but it can't see your VRAM.
#6169: What It's Like to Be a Token Inside an LLM
A token's-eye view of an LLM's forward pass — no cone, no squeezing, just a hallway where every layer scribbles on you.
#6165: Watching 20 Batch Jobs at Once
Batch APIs give you a discount, then hand you a babysitting job. Here's what actually watches a batch run — and what doesn't.
#6164: We Gave Each Host Its Own AI Agent
We split our two hosts into separate AI agents with private context windows. The bill came to 78 cents an episode.
#6162: Why Dictated Words Rewrite Themselves
Hold-to-talk dictation shows dimmed text that silently changes when you release the key. Here's the architecture behind that rewrite.
#6160: Where Do Tiny Speech Rules Live? A Ladder From Regex to LoRA
A regex notebook, a WFST grammar, or a fine-tune? Where do tiny speech-rewrite rules actually belong in a 600M-parameter model?
#6153: Decision Models: Typed Answers in One Forward Pass
Clef, Jev, and OpenAI's Decisions API return typed answers with probabilities in a single forward pass — no generated text to parse.
#6148: The Tiny Hebrew Models Behind the Show's Voice
Dicta's 300M-parameter nikudizer, Phonikud's adaptors, and the ktiv male model hiding inside a vowel adder.
#6146: Grounding Is Not Solved: DIY vs Gateway vs Native Search
Bolt-on search, gateway search, or native grounding? The plumbing didn't vanish — it just moved to someone else's bill.
#6139: Which AI Writes Our Scripts Best? We Tested Five
Five models, three scripts each, one rubric. Sol won on length, Grok on voice — and the runner-up title hides a catch.
#6135: How Many Runs Does Your AI Eval Actually Need?
Reruns shrink one kind of noise and leave another untouched — so where should your eval budget really go?
#6134: Cheap Tokens, Expensive Scripts
A model with worse list prices can cost less per script — if the cache hits. The bill hides in tokens, not rates.
#6132: How to Run a Model Eval on a Budget
Daniel is teaching himself to run evals on a real budget. Successive halving, paired comparisons, and why three clean scripts prove almost nothing.
#6127: Haiku 5.5 at 10 Cents: What the Cheap Model Actually Does
Haiku 5.5 costs 10 cents per million input tokens. We break down where it holds up, where it fails, and whether it can write this show.
#6104: Mistral AI : le pari européen de la souveraineté en intelligence artificielle
Comment Mistral AI défie la Silicon Valley avec des modèles efficaces, des partenariats stratégiques et son nouveau modèle vocal Voxtral ? Corn et Herman font le point.
#6092: 到底是谁在塑造中国的人工智能思想？
不只是实验室的老板：那些定义中国如何打造、又如何争论人工智能的研究者和思想。
#6076: How AI Actually Predicts Elections
Bayesian models, "silicon sampling," and agent societies — what actually works for forecasting votes, and what's just fashion.
#6048: Under the Hood, October 2026: Seventeen Prompt Versions and Three-Times-Faster Voices
Four weeks of pipeline changes, grounded in the code: grace periods, duplicate detection, Hilbert's rules, echo deletion, and voices nearly three times faster.
#6038: The Model You Pay For Isn't the Best Deal
Eight Chinese labs, two tiers each, and a pricing spreadsheet that says your default model is mid-pack.
#6037: Gemini 4 Argon: Google's Million-Token Flagship
A close read of Gemini 4 Argon's model card — a million-token window, thinking levels, implicit-only caching, and a lot of blanks.
#6036: GPT-6 Astra: A Million Tokens, No Fine-Tuning
A million-token window, ten built-in tools, a 272K pricing cliff — and no fine-tuning. What GPT-6 Astra actually offers builders.