Groq
Inference provider running models on custom chips
Episodes
-
#6001: What Do Inference Providers Actually Sell?Same open weights, 20x cost gap. What Fireworks, Together, and Groq actually sell — and when to just rent a GPU. -
#5981: Batch Inference: Four Doors, Half PriceSame tokens, half the price — and almost nobody using it properly. A tour of batch inference from OpenAI to OpenRouter. -
#5959: What RTF Actually Tells You About Speech ModelsAccuracy gets the headlines, but speed decides what ships. How RTF, latency percentiles, and memory shape real ASR deployments. -
#5469: Open Source's New Map: India, Croatia, EthiopiaOne in three new GitHub developers now comes from a country outside 2020's top ten. What that shift means for open source. -
#5004: What Precision Is Your Model Actually Running At?When you call an API, are you getting the model you think you are? Inference providers quantize more than they admit. -
#2432: The Hidden Cost of Flexibility in Chip DesignThe economics and engineering of ASICs vs. CPUs and GPUs, from transistor placement to hyperscaler strategy. -
#2431: The 3 Markets in an AI Trench CoatGPUs, LPUs, and ASICs: why the best hardware for AI depends entirely on what you're trying to do. -
#2227: Why Groq's Chip Flips the AI Hardware Script -
#2214: The Three Failure Modes of AI News SystemsWhen a conflict changes hourly, AI systems built for yesterday's information fail. Here's how to architect pipelines that actually keep up. -
#2183: Making Voice Agents Feel NaturalTurn-taking, interruptions, and latency are destroying voice AI UX—and the fixes are deeply technical. Here's what's actually happening underneath. -
#1946: Why LangChain Built a Three-Layer Agent StackWe unpack LangGraph, LangChain, and Deep Agents to reveal the deliberate hierarchy behind the ecosystem. -
#1723: Why Agentic AI Needs a Hive Mind, Not a Single BrainThe single monolithic AI model is dying. Meet the new native multi-agent architectures that think like a team, not a solo genius. -
#1216: AI Wearables: Local Sovereignty vs. The Subscription TrapDiscover the trade-offs between sleek AI subscriptions and open-source sovereignty. Can local processing save your data from the cloud? -
#1078: The Agentic Throughput Gap: Why Your AI Hits a WallStop hitting 429 errors. We explore why AI agents crash into rate limits and how to build high-throughput systems that never sleep. -
#874: From Vibes to Engineering: Mastering JSON Schema for AIStop begging your AI for clean data. Learn how JSON schema turns unreliable LLM responses into strict, predictable software components. -
#82: The Accidental AI EngineFrom video game dragons to digital brains: Herman and Corn explain why your graphics card is the secret engine behind the AI boom.