GPT-3

The classics all said 'learn it from data and scale it up'. This year's most-discussed AI papers are auditing what scaling took for granted.

Encoder, decoder, or both? The three transformer shapes, why one won, and why BERT still refuses to die.

The model doesn't choose tokens or decide when to stop — the engine does. A step-by-step tour of one decode iteration.

New capabilities seem to appear out of nowhere as models scale. Whether that's real — or just how we measure — is still an open fight.

LoRA lets you fine-tune an LLM’s behavior with a 50MB file. Here’s how it works and why it matters.

Daniel argues AI development should slow down for expertise and stability. But who in the industry actually shares this philosophy beyond the obvio...

The debate over whether AI models exhibit genuine emergent abilities or just appear to because of how we measure them—and why it matters for safety...

The "bigger is better" era of AI is over. Here's why the industry hit a data wall and shifted to a new scaling law.