gaia
General AI Assistants benchmark for agentic tasks
Episodes
-
#2178: How to Actually Evaluate AI AgentsFrontier models score 80% on one agent benchmark and 45% on another. The difference isn't the model—it's contamination, scaffolding, and how the te... -
#2174: Role-Playing as OrchestrationHow a role-playing protocol from NeurIPS 2023 became one of AI's most underrated agent frameworks—and what happens when you scale it to a million a... -
#130: How to Spot Gamed Benchmarks in Chinese AIAre Chinese AI models actually beating the West, or just gaming the system? Herman and Corn dive into the reality of modern AI benchmarks.