Yarn
Context extension technique for RoPE
Episodes
-
#5182: DeepSeek V4.1 Flash: 1M Context, 437x Smaller KV CacheDeepSeek V4.1 Flash landed with a 1M-token window and a KV cache 437x smaller than V1. Here's what actually changed — and why the middle of your co... -
#2062: How Transformers Learn Word Order: From Sine Waves to RoPETransformers can’t see word order by default. Here’s how positional encoding fixes that—from sine waves to RoPE and massive context windows. -
#1647: Monorepos: Better Modularity Than Multi-Repos?Why putting all your code in one repository can actually enforce better boundaries than separate repos.