RWKV
Recurrent architecture that trains like a transformer
Episodes
-
#6211: Attention Is Still Winning: The Hybrid Model CensusMamba layers shipped — inside hybrids. A census of Qwen, Nemotron, Kimi, Granite, and why MiniMax went back to full attention. -
#2672: When a Startup Claims to Break the Quadratic WallA startup claims linear attention scaling at 12M tokens, beating GPT-5.5 on retrieval benchmarks. -
#2065: Why Run One AI When You Can Run Two?Speculative decoding makes LLMs 2-3x faster with zero quality loss by using a small draft model to guess tokens that a large model verifies in para... -
#1081: The K-V Cache: Solving AI’s Invisible Memory TaxWhy does your AI get slower as you chat? Discover the K-V cache, the invisible bottleneck of generative AI, and how we're fixing it in 2026. -
#182: The Quadratic Bottleneck: Why AI Needs New ArchitecturesIs the transformer’s reign ending? Herman and Corn explore Mamba, x-LSTM, and the architectures solving AI's massive memory problem.