cuda
NVIDIA's parallel computing platform for GPUs
Episodes
-
#5991: Batch Inference on Your Own GPU: vLLM, SGLang, Ollama, llama.cppHosted batch APIs sell you half-price tokens. Here's how to get the same file-in, file-out workflow on your own Linux box. -
#5419: The Attention Budget in Your PocketWhy phone dictation runs out of room, and how bounded attention windows buy you punctuation without blowing your memory budget. -
#2779: The Hidden Stateful Side of Serverless GPUHow Modal, RunPod, and other platforms handle container builds, caching, and versioning under the hood. -
#2777: GPU Idle Waste and Serverless Green ComputingWhy your dedicated GPU burns 130 watts doing nothing, and how serverless platforms cut energy waste by more than half. -
#2432: The Hidden Cost of Flexibility in Chip DesignThe economics and engineering of ASICs vs. CPUs and GPUs, from transistor placement to hyperscaler strategy. -
#2227: Why Groq's Chip Flips the AI Hardware Script -
#1807: The ABI Trap: Why GPUs Break Docker's PromiseDocker promised "run anywhere," but GPU images make you compile for hours. Here’s why the abstraction breaks down. -
#1717: The AI Framework Name GameWhy are there thousands of "AI frameworks" on GitHub? We unpack the naming mess and the cost of semantic inflation. -
#1607: NVIDIA’s $26 Billion Pivot: From Chips to AI ModelsNVIDIA is moving beyond chips to build the "brains" of AI. Explore the $26B shift into models, robotics, and the new Rubin platform. -
#1227: Mojo 1.0: Can Chris Lattner Fix the AI Performance Gap?Explore how Mojo aims to unify Python’s ease of use with C++ performance to solve the "two-language problem" in AI development. -
#1224: Cracking the CUDA Code: NVIDIA’s Software DominanceDiscover why NVIDIA’s CUDA is the oxygen of the AI industry and how tools like OpenAI’s Triton are finally challenging its 20-year software moat. -
#1021: The Python Paradox: Why AI's Backbone Is a Nightmare to DeployWhy did a 1980s hobby project become the backbone of AI? Explore the history of Python and the chaos of modern dependency management. -
#170: How PyTorch Beat TensorFlow and Became AI's BackboneDiscover why PyTorch is the "oxygen" of AI. Herman and Corn explore its history, the magic of Autograd, and the move to the PyTorch Foundation. -
#84: The Silicon Arms Race: Why GPUs are the New OilAre high-end microchips the new enriched uranium? Herman and Corn dive into the high-stakes world of GPU export bans and global AI supremacy. -
#82: The Accidental AI EngineFrom video game dragons to digital brains: Herman and Corn explain why your graphics card is the secret engine behind the AI boom. -
#34: Red Team vs. Green: Local AI Hardware WarsNVIDIA's CUDA rules AI, leaving AMD users battling a "green wall." Explore the hardware wars and thorny paths forward. -
#25: The Open-Source Battle for AI's Hardware SoulUnraveling how GPUs power AI. We dive into CUDA, ROCm, and the software stack that makes it all think. -
#11: How Does Fine Tuning Work Anyway?Unlock the secrets of AI fine-tuning. Discover how your small dataset can shape a giant model. -
#4: If Your Voice Ages, Does Your Fine-Tune Become Useless?Your voice changes, but your fine-tuned model shouldn't become useless. We explore the biology of the larynx and ASR. -
#9: Benchmarking Custom ASR Tools - Beyond The WERBenchmarking custom ASR fine-tunes: We're diving deep beyond the WER to truly measure performance.