GGUF
Quantized model file format for llama.cpp
Episodes
-
#6191: What Actually Happens During Fine-TuningSixteen bytes per parameter in, two out — a walk through the forward pass, backward pass, and optimizer that turn a dataset into new weights. -
#5991: Batch Inference on Your Own GPU: vLLM, SGLang, Ollama, llama.cppHosted batch APIs sell you half-price tokens. Here's how to get the same file-in, file-out workflow on your own Linux box. -
#5416: Xet, Buckets, and Auto-Pulling Model WeightsHugging Face swapped its storage backend to Xet with barely a ripple. So why can't a bucket auto-pull new upstream weights? -
#2017: The Art of Squeezing AI Models onto Your GPUThose cryptic letters on Hugging Face actually map how much brain power you trade for speed. -
#1943: The Invisible Math Shrinking AI ModelsLZMA, Zstandard, and Brotli are shrinking massive AI models, but how do they actually work? -
#1544: Why It Costs More to Run AI Than to Build ItDiscover why the AI runtime is the unsung hero of the tech stack, determining whether your AI feels like a snappy conversation or a slow crawl. -
#41: The Butcher's Bargain: When Smaller AI Is Good EnoughUnlock powerful AI on your device! We demystify quantization, the ingenious trick making local AI a reality. -
#8: Building Your Own WhisperEver wondered if you could build your own speech recognition tool? We dive deep into crafting custom ASR. -
#4: If Your Voice Ages, Does Your Fine-Tune Become Useless?Your voice changes, but your fine-tuned model shouldn't become useless. We explore the biology of the larynx and ASR. -
#6: How To Fine Tune WhisperBuild your own AI transcription tool! We'll walk you through fine-tuning Whisper, from data to notebook. -
#7: Building Custom ASR ToolsEver wondered how to build your own ASR tools from scratch? Discover the why and how in this episode! -
#3: Safetensors or something else: STT inference formats explainedUnpacking ASR weight formats: Safetensors and beyond. Tune in to understand the distinctions. -
#5: Fine-Tuning ASR For Maximal UsabilityFine-tuned ASR is just the start. Discover the next steps for deployment and maximizing usability. -
#10: How ASR Went From Frustration To ... Whisper MagicSpeech to text: from frustrating to fantastic. Uncover the magic behind its rapid rise and connection to the AI boom! -
#2: Local STT For AMD GPU OwnersAMD GPU? No problem! Dive into local AI adventures like on-device speech to text. -
#11: How Does Fine Tuning Work Anyway?Unlock the secrets of AI fine-tuning. Discover how your small dataset can shape a giant model.