DeepEval
Open-source framework for LLM evaluation
Episodes
-
#3284: Agent Infrastructure Engineer: The New DevOpsAgentic AI is splintering into real engineering disciplines. Here's what the "DevOps of AI" actually does. -
#2403: Choosing Your LLM Eval FrameworkAn architectural shootout of four major LLM evaluation harnesses — where each shines and where each breaks down. -
#2005: Beyond Vibes: The Hard Science of LLM EvaluationRunning the same LLM on different GPUs can produce different results. Here’s why that happens and how to test for it. -
#1929: From Vibe Checks to Model MetricsWe stopped "vibe-checking" our AI scripts and built a science fair for models. Here's how we grade them. -
#1762: Testing AI Truthfulness: Beyond VibesStop trusting confident AI. We explore the formal science of testing LLMs for hallucinations and knowledge cutoffs. -
#1472: Stop Flying Your AI Agents BlindMove past basic token counting. Learn how to monitor AI reasoning, prevent $47k loops, and build trust in autonomous agents.