AI Inference
549 episodes · Page 15 of 23
#2063: That $500M Chatbot Is Just a Base Model
That polite chatbot? It started as a raw, chaotic autocomplete engine costing half a billion dollars to build.
#2062: How Transformers Learn Word Order: From Sine Waves to RoPE
Transformers can’t see word order by default. Here’s how positional encoding fixes that—from sine waves to RoPE and massive context windows.
#2061: The Memory Bottleneck That Drives Attention Design
Attention is the engine of modern AI, but it’s also a memory hog. Here’s how MQA, GQA, and MLA evolved to fix it.
#2060: The Tokenizer's Hidden Tax on Non-English Text
Why does a simple greeting in Mandarin cost more to process than in English? It's the tokenizer's hidden inefficiency.
#2056: Music as Language: The Architecture Behind AI Song Generation
A look at how AI music models use audio tokens, transformers, and diffusion to turn text into songs.
#2041: The "MPEG Moment" for AI: Llamafile & Native Models
Why are we squeezing massive cloud models onto desktops? Meet the "native" AI revolution.
#2040: The Rebellion Against Big Tech's AI Lock-In
Why run LLMs locally? We break down Ollama, llama.cpp, vLLM, and llamafile—and when to use each.
#2022: When AI Becomes Your IT Department
We dug into a repo of 47 real-world projects showing how OpenClaw powers everything from self-healing servers to overnight app builders.
#2021: Your Frozen AI Is Getting Smarter (Here's How)
Your AI model might be static, but the system around it can make it learn in real-time.
#2019: Local AI vs Cloud AI: The Agent Identity Crisis
Your desktop is becoming a life support system for AI agents. We explore the sharp trade-offs between local-first and cloud-native architectures.
#2017: The Art of Squeezing AI Models onto Your GPU
Those cryptic letters on Hugging Face actually map how much brain power you trade for speed.
#2012: Pixels vs Protocols: The Computer Use Showdown
Is visual AI a bridge or the future? We debate the efficiency and longevity of "Computer Use" agents versus API-first automation.
#2009: The Plumbing of AI Safety: Guardrails, Not Vibes
We dive deep into the specific libraries, proxy layers, and architectural decisions that keep an LLM from emptying a bank account.
#2005: Beyond Vibes: The Hard Science of LLM Evaluation
Running the same LLM on different GPUs can produce different results. Here’s why that happens and how to test for it.
#2004: The AI Control Plane Is Here (But Is It Safe?)
Your LLM, tools, and costs are scattered across dashboards. Here’s how a unified AI control plane fixes the chaos.
#1992: The Sovereign Compute Shift: Owning vs. Renting AI Iron
Israel is building a sovereign AI supercomputer with 4,000 Nvidia B200 GPUs to keep startups local.
#1987: Can You Ever Quit Your Personal AI?
Your AI knows your workflow, but can you ever leave? We explore the lock-in risks of personal AI agents.
#1964: The Three Layers That Make AR Finally Work
See a 3D arrow pointing to the exact bolt you need, or read a street sign in real-time translation.
#1962: Moravec's Paradox: Why Robots Can Write Poetry but Can't Fold a Fitted Sheet
We explore the tech letting robots "reason" about physical tasks using vision-language-action models.
#1946: Why LangChain Built a Three-Layer Agent Stack
We unpack LangGraph, LangChain, and Deep Agents to reveal the deliberate hierarchy behind the ecosystem.
#1943: The Invisible Math Shrinking AI Models
LZMA, Zstandard, and Brotli are shrinking massive AI models, but how do they actually work?
#1940: Why Google's 31B Model Fits in Your GPU
Google just dropped Gemma four, and its 31-billion-parameter size is a masterclass in hardware-aware AI design.
#1931: Where Your AI Pipeline Actually Dies
Why do AI pipelines crash? It’s not the models—it’s the plumbing. We break down how to manage data between stages.
#1930: The Agent Identity Crisis: Workflow vs. Conversation
One automates invoices silently; the other chats in Slack. Why the industry's favorite word means two totally different things.