FlashAttention
Fused attention kernel hiding attention patterns
Episodes
-
#6024: What's Actually Inside the KV Cache?890 bytes per token: why the KV cache is both what makes LLMs practical and what makes long context expensive. -
#6013: What Happens Inside One LLM Forward PassOne trip through the network, layer by layer — and why a classifier needs one pass while an LLM needs hundreds. -
#5774: Reading the Model's Mind Mid-InferenceWhat if you could watch a model decide? Inside the tools that open up inference mid-computation — and the limits they hit. -
#2062: How Transformers Learn Word Order: From Sine Waves to RoPETransformers can’t see word order by default. Here’s how positional encoding fixes that—from sine waves to RoPE and massive context windows. -
#1111: Surviving the arXiv Deluge: Finding Signal in AI's Paper FirehoseDiscover the unsung research papers that built the AI era and learn how to navigate the relentless flood of new machine learning breakthroughs. -
#126: The Cocktail Party Problem: Why AI ForgetsWhy do AI models "lose the plot" after a few thousand words? Discover the mechanics of attention and the innovations solving context window limits.