Thursday, 12 November 2026
3 episodes · 6596 in all
#6540: Serving Parakeet Speech Models on Cheap CPUs
Quantized INT8 Parakeet speech models can run up to fifty-four times real-time on standard eight-core CPUs without touching a GPU.
#6539: When Real Engineering Teams Still Choose Encoders
Roblox achieved over 3,000 inferences per second on a 36-core Xeon CPU with BERT, outperforming equivalent GPUs by six to one.
#6538: How Models Score Next Tokens All at Once
A model doesn't loop over vocabulary words to pick the next one: a single matrix multiplication computes scores for all candidates at once.