SWE-bench Pro

Harder version for non-memorized tasks

3 episodes mention this Visit official site →

Episodes

#6278: Why Model Benchmarks Miss Small Workflow Friction

Switching Claude Code to Sonnet broke an everyday shell alias request that Opus solved simply by recognizing where to look in local context.

Oct 25
#1831: The 79% AI Coder: Reasoning vs. Memorization

AI models now score 79% on coding benchmarks, but a 40-point drop on harder tests reveals the truth.

Mar 31
#1464: When AI Agents Author 4% of All Code

Explore how agentic harnesses transform AI from a passive chatbot into an active developer capable of full-cycle software engineering.

Mar 23