LangSmith
Observability and tracing for LLM applications
Episodes
-
#5623: When Your AI Cites Sources It Didn't Actually UseYour RAG pipeline cites a source — but did the model actually use it? New interpretability research says you can't tell from the trace. -
#3284: Agent Infrastructure Engineer: The New DevOpsAgentic AI is splintering into real engineering disciplines. Here's what the "DevOps of AI" actually does. -
#2924: When Adding One Agent Breaks EverythingThe math behind why your 100-agent pipeline fails 40% of the time — and what to do about it. -
#2780: Building Self-Healing Agent PipelinesHow to build an agent that monitors and fixes other agents in production — without the hype. -
#2773: Beyond Static Fallbacks: Agentic Error Handling in AI PipelinesFrom try-except blocks to planning agents that route around failures intelligently. -
#2254: How to Test an AI Pipeline ChangeWhen you tweak one part of a complex AI agent system, how do you know if it actually improved anything? The answer lies in engineering checkpoints. -
#2213: When Ground Truth Moves HourlyHow do you rigorously evaluate whether Tavily or Exa retrieves better results for breaking news? A formal benchmark beats the vibe check. -
#2185: Taking AI Agents From Demo to ProductionSixty-two percent of companies are experimenting with AI agents, but only 23% are scaling them—and 40% of projects will be canceled by 2027. The ga... -
#2184: The Economics of Running AI AgentsProduction AI agents can cost $500K/month before optimization. Learn model routing, prompt caching, and token budgeting to cut costs 40-85% without... -
#2178: How to Actually Evaluate AI AgentsFrontier models score 80% on one agent benchmark and 45% on another. The difference isn't the model—it's contamination, scaffolding, and how the te... -
#2169: How Enterprises Are Rethinking Agent FrameworksTwelve major agentic AI frameworks exist—yet many serious developers avoid them entirely. What patterns emerge in real enterprise adoption? -
#2166: Code vs. Canvas: How Developers Pick Their ToolsLangGraph or Flowise? The honest answer isn't obvious. Developers gain speed and integrations with visual builders—but lose version control, testin... -
#2165: Strip Your Agent to BashThe frameworks matter less than you think. What separates a working agent from a failing one is the harness—the orchestration, memory, and tool des... -
#2010: Building Better AI Memory SystemsWe obsess over AI inputs but treat outputs like Snapchat messages. Here's why that's a massive blind spot. -
#1946: Why LangChain Built a Three-Layer Agent StackWe unpack LangGraph, LangChain, and Deep Agents to reveal the deliberate hierarchy behind the ecosystem. -
#1939: API Drift and Agent ReliabilityWhen an API changes without warning, your AI agent can crash spectacularly. Here's how to test the new "plumbing" of the agentic age. -
#1929: From Vibe Checks to Model MetricsWe stopped "vibe-checking" our AI scripts and built a science fair for models. Here's how we grade them. -
#1870: Learning to Break Things SafelyLearn how to safely build and test autonomous AI agents using a disposable VPS, Docker containers, and secure networking. -
#1804: The Fork in the Road: Why AI Agents Check Old Receipts FirstStop your AI agent from overthinking. Learn why it checks old memories instead of booking flights—and how to fix the "eagerness" problem. -
#1472: Stop Flying Your AI Agents BlindMove past basic token counting. Learn how to monitor AI reasoning, prevent $47k loops, and build trust in autonomous agents. -
#1235: Beyond "No Training": Securing the New Agentic AI StackThink your data is safe because of a "no training" clause? We deconstruct the hidden security risks within the modern agentic AI stack. -
#1083: Mapping the Second Black Box: Agentic AI VisualizationStop reading messy logs. Discover how mapping "internal momentum" and latent value spaces can solve the black box problem in agentic AI. -
#938: From Hobbyist Scripts to Agent InfrastructureStop building brittle bots. Learn how to scale and maintain complex AI agent workflows using the new generation of open-source orchestration tools. -
#841: AI Gateways: Building Robust Infrastructure with LiteLLMDiscover how AI gateways like LiteLLM provide redundancy, caching, and unified tool access for scalable application development.