GPT-2

Early language model from OpenAI

4 episodes mention this Visit official site →

Episodes

#6368: What Does It Take To Train An AI Model From Scratch?

nanochat trains a BPE tokenizer in Rust in roughly 34 seconds, yet it can quietly disadvantage languages like Korean by nearly half.

Oct 30
#6014: Why Every AI Model Is Secretly a Decoder

Encoder, decoder, or both? The three transformer shapes, why one won, and why BERT still refuses to die.

Oct 15
#1574: Weird AI Experiment: The Arrogance Interview

Can an AI feel pride? Watch what happens when two versions of the same model face off in a high-stakes interview about digital superiority.

Mar 26
#260: Digital Archeology: The Primitive Power of GPT-1

Revisit the 2018 model that started it all. Herman and Corn dive into GPT-1's romance-novel roots and its 117-million-parameter legacy.

Jan 20