MMLU-Pro
Reasoning benchmark used in fragility study
Episodes
-
#5673: Why Lowering Temperature Broke Daniel's ScriptsLowering temperature should make output safer, not incoherent. Daniel's DeepSeek scripts say otherwise — and the reason may be stranger than he thi... -
#5169: Fine-Tuning vs From-Scratch for Minor LanguagesWhat 774 experiments and a new Armenian model reveal about the tradeoff between fluency and knowledge in low-resource languages. -
#3157: Opus 4.8: What Actually Changed Under the HoodAnthropic dropped Opus 4.8 with no fanfare. New training data, faster inference, and smarter refusals — here's what changed. -
#1231: The Agentic Shift: 5 Bold AI Predictions for 2026The Poppleberry brothers move past the chatbot era to deliver five high-stakes, falsifiable predictions for the future of autonomous AI agents. -
#651: Decoding the Blueprint: An Expert Guide to AI Model CardsStop skipping the fine print. Herman and Corn reveal how to read AI model cards like a pro to spot true innovation and hidden flaws.