llama 2
Meta's open-source LLM
Episodes
-
#6024: What's Actually Inside the KV Cache?890 bytes per token: why the KV cache is both what makes LLMs practical and what makes long context expensive. -
#6012: Inside the Decode Loop: How LLMs Generate One Token at a TimeThe model doesn't choose tokens or decide when to stop — the engine does. A step-by-step tour of one decode iteration. -
#4966: Local vs Cloud: Running Hugging Face ModelsHugging Face's compatibility tracker, cache management, and the real difference between Inference Endpoints and the direct API. -
#2494: Active Prompt Engineering: Daniel's Diff-Based LoopA deep dive into iterative prompt refinement using inter-iteration prediction change as an uncertainty signal. -
#2357: Microsoft's Phi: When Data Quality Beats Model SizeExplore Microsoft AI's Phi family of small language models, designed for edge deployment and high efficiency. -
#2061: The Memory Bottleneck That Drives Attention DesignAttention is the engine of modern AI, but it’s also a memory hog. Here’s how MQA, GQA, and MLA evolved to fix it. -
#1754: From Ollama to Agentic CLIs: The Rise of the AI HarnessExplore the evolution from local LLMs to modern agentic CLIs, focusing on the "harness" that gives models context, tools, and autonomy. -
#1698: Can AI Models Represent Nations in Diplomacy?Real projects are building AI agents trained on national laws and diplomatic archives to simulate negotiations. -
#1099: Digital Recalls: Why Your AI Is Losing Its EdgeIs your AI getting lazier? Explore the "digital recall" and why the world’s most advanced models are secretly taking steps backward.