Ollama
Tool for running local LLMs
Episodes
-
#5991: Batch Inference on Your Own GPU: vLLM, SGLang, Ollama, llama.cppHosted batch APIs sell you half-price tokens. Here's how to get the same file-in, file-out workflow on your own Linux box. -
#5969: Ubuntu 26.04 LTS: Is Staying on LTS Still a Trade-Off?Ubuntu 26.04 LTS shipped in April. Point releases, HWE kernels, and SRUs have quietly closed most of the freshness gap — but not everywhere. -
#5899: Same GPU, Same Weights, Different AnswersIdentical hardware, identical weights, different engine — and answers that shift by up to 16.6 points. -
#5898: ROCm 10.1: AMD's Software Stack Finally Grows UpThree ROCm releases in six months — build system, agentic tooling, data movement. What actually changed for AMD GPU owners. -
#5853: Who Actually Competes With Hugging Face?2.96 million model repos, and 85.6% have under 200 downloads. What ModelScope, Kaggle, and Ollama actually do differently. -
#5436: Small Models as Rewriters, Not WritersWhy "don't say X" prompts backfire, and how a tiny grammar-constrained model can scrub a script without breaking its grammar. -
#5405: Omarchy: The Linux Distro Built for AI AgentsOmarchy treats AI agents as users of the OS itself — every setting a command, every config a text file. Here's how it works and where it breaks. -
#5393: Fine-Tuning a Model on 100 Hand-Edited AnswersYou don't need 10,000 examples to make a model sound like you. The real number is closer to 100 — if the edits are opinionated. -
#4973: Building a Pure AI Inference ServerWhy your "AI server" should look more like a database than a web app, and how to actually build one. -
#4972: CPU vs GPU Inference: When to Skip the GPUWhy running AI on CPU isn't a toy — and when the GPU advantage vanishes. -
#4778: SSH MCP and Tailscale: Docker Migration GuideHow to configure Tailscale for Docker containers when migrating from public IPs to tailnet addresses. -
#4588: The Hidden Cost of Conversational AI: Why Stateless LLM Calls MatterWhy conversational AI tools are making batch document processing slower, costlier, and less reliable — and what to do about it. -
#4455: Borrowing a GPU Over Your Home NetworkCan you use your desktop's AMD GPU from a laptop as if it were local? We explore the real options. -
#4450: Intercepting Claude Code: The Proxy EcosystemHow middleware proxies intercept, reroute, and transform AI tool traffic — and what it means for users. -
#4296: Auto-Splitting AI Threads: Smarter Context ManagementHow to build a system that automatically detects topic shifts and routes messages to the right thread. -
#4181: From Phone Typing to Direct Agent: Fixing Headless Server AI AccessChatGPT saved his OS, but the copy-paste bottleneck from phone to server took hours. Here's how to fix it. -
#4139: The Hosepipe-to-Digest AI: Your Outbox MiddlewareAn AI middleware that intercepts your outgoing emails, buffers them, and delivers a crisp executive summary. -
#3322: Desk Pets: Local AI on Your DesktopAre desk pets useful tools or expensive Tamagotchis? We break down the local AI, privacy tradeoffs, and psychology behind these devices. -
#3271: LLMs as Parsers, Not CalculatorsStop letting LLMs do math. Use them to parse messy text, then let deterministic code handle the numbers. -
#2688: Intelligent Frame Extraction for Multimodal AIUse multimodal AI and smart frame extraction to turn a walk-through video into an actionable decluttering plan. -
#2536: Self-Hosted Zapier Alternatives in 2026n8n, Huginn, and Dagu compared for personal automation on your own hardware. -
#2470: Where Intelligence Should Live in Your PipelineWhen should you fine-tune a tiny model for prompt enhancement instead of prompting a large one? The answer depends on latency, precision, and domain. -
#2361: The Cognitive Load of LogsHow Claude Code transforms Linux system administration by automating log analysis and proactive maintenance. -
#2357: Microsoft's Phi: When Data Quality Beats Model SizeExplore Microsoft AI's Phi family of small language models, designed for edge deployment and high efficiency. -
#2355: Why Open-Weight Models Are WinningDiscover how Cogito v2.1 leverages process supervision and MoE architecture to redefine reasoning efficiency in open-weight AI models. -
#2271: Vector Search in a Single FileWhat if you could do vector search with just SQLite? We explore sqlite-vec, the extension that adds embeddings to the world's simplest database, an... -
#2175: Let Your AI Argue With ItselfWhat happens when you let multiple AI personas debate each other instead of asking one model one question? A deep dive into synthetic perspective e... -
#2040: The Rebellion Against Big Tech's AI Lock-InWhy run LLMs locally? We break down Ollama, llama.cpp, vLLM, and llamafile—and when to use each. -
#2038: UI-First vs Architecture-First: Choosing Your AI AgentLobeHub vs. Dify vs. n8n: We break down the chaotic landscape of local AI agents to find the right "brain" for your workflow. -
#2027: The Missing Photoshop for WordsWhy is editing text with AI so clunky? We explore the "TITO" paradigm—using small, local models for fast, private text transformation. -
#2019: Local AI vs Cloud AI: The Agent Identity CrisisYour desktop is becoming a life support system for AI agents. We explore the sharp trade-offs between local-first and cloud-native architectures. -
#2017: The Art of Squeezing AI Models onto Your GPUThose cryptic letters on Hugging Face actually map how much brain power you trade for speed. -
#1940: Why Google's 31B Model Fits in Your GPUGoogle just dropped Gemma four, and its 31-billion-parameter size is a masterclass in hardware-aware AI design. -
#1906: Is Your AI Model Agentic-Ready or Just Wearing a Suit?Native tool calling is the difference between a working product and a debugging nightmare. -
#1764: Your Repo as a Knowledge BaseHow to give AI agents instant memory of your entire project—without cloud costs or complex infrastructure. -
#1754: From Ollama to Agentic CLIs: The Rise of the AI HarnessExplore the evolution from local LLMs to modern agentic CLIs, focusing on the "harness" that gives models context, tools, and autonomy. -
#1716: Seeing the AI Think: Visual Debugging for Agent WorkflowsSee how a visual, node-based tool lets you build complex AI agent workflows without writing code. -
#1620: Why VRAM Is the Wrong Way to Measure Your AI PCForget VRAM—bandwidth is the new king. Discover why your local AI feels slow and how to build a true "agent computer" for professional coding. -
#1544: Why It Costs More to Run AI Than to Build ItDiscover why the AI runtime is the unsung hero of the tech stack, determining whether your AI feels like a snappy conversation or a slow crawl. -
#1407: Can One Million LLMs Predict the Next Global Crisis?Discover how an undergraduate student built a viral simulation of one million AI agents to predict social behavior and policy outcomes. -
#1078: The Agentic Throughput Gap: Why Your AI Hits a WallStop hitting 429 errors. We explore why AI agents crash into rate limits and how to build high-throughput systems that never sleep. -
#1077: Will Your Browser Replace Your OS for Local AI?See how Web GPU and Web NN are turning your browser into a local AI engine, ending the era of complex DIY setups and protecting your privacy. -
#1073: Beyond YAML: Building the Agentic Smart HomeStop wrestling with YAML. Discover how MCP and local AI agents are transforming Home Assistant into a truly intelligent, aware partner. -
#980: The Rosehill Audit: Mapping a Digital FootprintFrom Linux automation to AI prompts, discover the digital blueprint of a modern systems builder in this deep-dive investigative audit. -
#938: From Hobbyist Scripts to Agent InfrastructureStop building brittle bots. Learn how to scale and maintain complex AI agent workflows using the new generation of open-source orchestration tools. -
#857: The Cognitive Cost of CapitalizationCan local AI fix your messy typing in real-time? Explore the tech behind "transparent buffers" that turn sloppy drafts into polished prose. -
#633: Memory Wars: The Future of Local Agentic AICan your PC handle the next wave of AI agents? Herman and Corn dive into VRAM, quantization, and the future of running LLMs locally. -
#440: Beyond the Diaper Log: From Tracking to Understanding Your Baby's BrainMove past the spreadsheet. Discover how AI tools provide deep neurological insights into your baby’s development and the "mental leaps" at seven mo... -
#265: Stop Memorizing Syntax and Start Describing ResultsStop memorizing complex syntax. Explore how AI agents are transforming the terminal into a natural language "Intent-Based Interface." -
#110: Why Agentic AI Needs More VRAM Than You ThinkLearn how to build a high-performance local AI server for agentic coding, from dual-GPU PC builds to the power of Mac's unified memory.