BIG-bench

A pelican on a bicycle, a badger for Groundhog Day, and a benchmark where passing is the bad outcome — the strangest AI evals ever run.

New capabilities seem to appear out of nowhere as models scale. Whether that's real — or just how we measure — is still an open fight.

The debate over whether AI models exhibit genuine emergent abilities or just appear to because of how we measure them—and why it matters for safety...