AI Evaluation
517 episodes · Page 11 of 22
#2640: Why Instructional Models Beat Conversational for Batch AI
Beyond cheaper tokens—how batch inference changes AI workflows and why instructional models beat conversational ones for automated jobs.
#2639: The Hidden Layer That Makes Search Work
Why your search results miss the mark — and how cross-encoders fix it.
#2612: Can You Speed-Date Your Way to the Right Therapy?
Finding the right therapy is a guessing game. What if you could sample different approaches before committing?
#2609: Mapping the Therapy Family Tree: CBT, ACT, DBT & Beyond
How CBT, ACT, and DBT actually evolved — and why matching therapy to personality matters.
#2586: Pseudo-Personalized Emails: The New Spam Uncanny Valley
How to detect and filter AI-generated outreach emails that fake personal connection without nuking legitimate messages.
#2578: Building Deliberately Slow Deployment Pipelines
How to build CI/CD pipelines designed as filters, not firehoses — with manual gates, staging environments, and quality checks.
#2551: How Progressive Disclosure Saves MCP from Token Bloat
Why dumping all tool schemas into context breaks accuracy — and three implementations that fix it.
#2549: Jakob's Law: Why Users Think Your App Is Broken
Why broken keyboard shortcuts destroy user trust — and what Jakob's Law reveals about design expectations.
#2544: How to Make AI Architectural Renders Photoreal Without Breaking Geometry
Fixing the uncanny valley in AI-enhanced architectural renders — without breaking the geometry.
#2539: When Does AI Stop Hallucinating and Start Reconstructing?
What happens when you feed hundreds of photos into an AI world generator — do you capture reality or just a convincing dream?
#2516: Overfitting Is Not a Binary Condition
Overfitting isn't binary. Learn the real triggers, the bias-variance tradeoff, and modern techniques to prevent it.
#2511: Measuring AI API Latency Through the Black Box
How to benchmark token throughput and debug slowdowns in closed CLI tools like Claude Code using OpenTelemetry and mitmproxy.
#2510: The Design That Makes Voice Agents Tolerable
Drive-thru accuracy, healthcare triage, and the design secret that makes people *want* to talk to a machine.
#2494: Active Prompt Engineering: Daniel's Diff-Based Loop
A deep dive into iterative prompt refinement using inter-iteration prediction change as an uncertainty signal.
#2483: Substitution Anonymization: Privacy Without Utility Loss
How to generate realistic synthetic voice notes and calendar data with zero PII exposure risk.
#2471: Creative Briefs for AI Agents: What Agencies Already Know
How agency best practices for briefing creatives map directly onto getting reliable output from AI agents like Claude Design.
#2468: When Tokens Meet GPU Seconds
How to track AI spend across Open Router, Replicate, and more — without a unified dashboard.
#2443: How Podcast RSS Feeds Can Speak Every Language
One RSS feed, a transcript tag, and TTS voice cloning — the emerging standard for letting any podcast speak any language.
#2419: Methylation vs. IEMs: Untangling the Confusion
Methylation isn't a health dial. Learn how it actually works in the body vs. rare genetic IEMs.
#2413: When Your AI Says No to Everything
Why LLMs refuse 73% of harmless prompts — and the trade-off between safety and usefulness.
#2412: When AI Caves: Progressive vs. Regressive Sycophancy
Why do LLMs agree with you even when you're wrong? We break down the SycEval benchmark and the 78% persistence problem.
#2411: Are Political Bias Benchmarks Actually Measuring Anything?
Why the Political Compass Test fails, and what researchers are building instead to actually measure model bias.
#2410: How Researchers Actually Measure Censorship in Chinese LLMs
Beyond headlines: the actual benchmarks, methodologies, and pitfalls in detecting political refusal in Chinese language models.
#2409: When AI Cheats on Cultural Knowledge
Five benchmarks that reveal how AI systems fail at cultural knowledge — and what their methodologies tell us.