AI Evaluation
517 episodes · Page 15 of 22
#2115: Why AI Answers Differ Even When You Ask Twice
You ask an AI the same question twice and get two different answers. It’s not a bug—it’s physics.
#2113: Goldfish vs Elephant: The Stateful Agent Dilemma
Stateless agents are cheap and fast, but stateful ones remember your window seat. Which architecture wins?
#2089: Open-Source vs. Military ATR: The Drone Recognition Gap
A public GitHub model spotted by a listener reveals the massive gap between hobbyist AI and lethal military drone detection systems.
#2076: Is Pure NLP Dead? The Hidden Scaffolding of AI
Modern AI didn't appear from nowhere. Discover how decades of linguistic rules and statistical models built the foundation for today's LLMs.
#2074: Generative Social Science: When AI Agents Develop Theory of Mind
See how a new framework models 10,000 virtual citizens to test policies before spending a dime.
#2069: The Vibe Coding Trap: Why Your Agent Skills Keep Breaking
Stop guessing at the agentskills.io spec. Learn the exact YAML fields, directory structure, and authoring patterns to make Claude Code skills that ...
#2068: Is Safety a Filter or a Feature?
External filters vs. baked-in ethics: the architectural war for LLM safety.
#2054: From Dirt to Data: How Empires Conquered the Cloud
Why did we stop conquering land and start conquering servers? This episode traces the shift from soil to bits.
#2025: How Do You Reward a Thought?
Rewarding an AI agent is harder than just saying "good job"—here's how we turn messy human values into math.
#2024: Your AI Council: Digital Committee or Groupthink?
A digital boardroom of AI models promises better decisions, but risks amplifying the same old biases.
#2021: Your Frozen AI Is Getting Smarter (Here's How)
Your AI model might be static, but the system around it can make it learn in real-time.
#2020: 1,000 AI Agents Built a Religion in Minecraft
An experiment drops 1,000 autonomous agents into Minecraft, and they spontaneously invent religion, democracy, and taxes.
#2016: Andrej Karpathy: The Bob Ross of Deep Learning
Why the most influential AI mind prefers a blank text file to proprietary black boxes.
#2010: Building Better AI Memory Systems
We obsess over AI inputs but treat outputs like Snapchat messages. Here's why that's a massive blind spot.
#2008: Needle-in-a-Haystack Testing for LLMs
New AI models claim to be genius-level, but can they actually find a specific fact in a massive document?
#2007: AI Grading AI: The Snake Eating Its Tail
We asked an AI to write this script. Then we asked another AI to grade it. Here’s what happens when the judges have biases.
#2006: How Do You Measure an LLM's "Soul"?
Traditional benchmarks can't measure tone or empathy. Here's how to evaluate if an AI model truly "gets it right."
#2005: Beyond Vibes: The Hard Science of LLM Evaluation
Running the same LLM on different GPUs can produce different results. Here’s why that happens and how to test for it.
#1994: Why Can't AI Admit When It's Guessing?
Enterprise AI now auto-filters low-confidence claims, but do these self-reported scores actually mean anything?
#1985: AI Tutors vs. Human Error: Who Do You Trust?
AI gets flak for hallucinations, but humans misremember 40% of facts. Why the double standard?
#1979: When Marketing Swallows the Tech
Is AI the same as Machine Learning? We break down the nested hierarchy of artificial intelligence, from symbolic logic to neural networks.
#1961: Weaponizing Your Weirdness in an AI World
As AI homogenizes the web, contrarian thinking becomes a scarce asset. Here’s how to weaponize your weirdness for a competitive edge.
#1959: How Constrained AI Models Handle the Unexpected
Your AI assistant promised to only use your documents. Instead, it invented a case law that doesn't exist. Here's why.
#1947: Curation Is the New Creation
With 47 new AI video tools launching in a week, finding the right one is harder than using it.