AI Inference
549 episodes · Page 7 of 23
#5773: Declaring Environments: Compose, Nix, and the Agent Boundary
Docker Compose, Nix, devcontainers, microVMs — how automated provisioning works, and where the boundary moves when an agent needs a workspace.
#5768: Who Actually Draws the Follow-Up Chips?
Those suggestion chips aren't the model talking — they're a second, cheaper inference call plus a rules-based classifier.
#5697: Decision Models That Return Probabilities, Not Text
A new class of model skips text generation entirely and returns calibrated probabilities in one forward pass. Here's what that changes.
#5673: Why Lowering Temperature Broke Daniel's Scripts
Lowering temperature should make output safer, not incoherent. Daniel's DeepSeek scripts say otherwise — and the reason may be stranger than he thi...
#5624: One Forward Pass: Building a Bounded Classifier
No generation, no parsing — just label scores in a single pass. How to build a small encoder classifier that actually works.
#5621: Constrained Decoding for AI-Generated Podcast Tags
A working AI podcast pipeline has one quiet failure: the tagging. Here's how constrained decoding and a stable taxonomy fix it.
#5573: Audio Post-Processing for AI Podcast Pipelines
Silence trimming, loudness normalization, EQ, and tempo fixes — the CLI tools that turn raw TTS output into a finished episode.
#5524: Why AI Voice Agents Can't Hang Up
Two polite AI agents, one endless goodbye loop — and a phone bill that never stops climbing.
#5507: Inside ChatGPT's Hidden Pipeline: 8 Models Per Reply
One chat turn isn't one model call. It's moderation, PII filtering, a tone-based router, and activation classifiers — here's the documented graph.
#5506: AI Is Way More Than Chatbots
The word "AI" quietly became a synonym for "chatbot." Here's what the full model ecosystem actually contains.
#5503: Inside the Hidden Image Generation Pipeline
That one-click image generator is secretly a graph of many models. We reconstruct the hidden pipeline behind Gemini and ChatGPT.
#5480: Writing a Personal Instruction for ChatGPT
Daniel's personal ChatGPT prompt gets a line-by-line critique — and the case for shorter, sharper custom instructions.
#5464: Your Keyboard's Hidden Data Problem
Your keyboard knows your email address, your phrases, your habits — and you can't take any of it with you.
#5460: Four Small Models, One Android Phone: Does It Actually Work?
A chained on-device dictation pipeline — VAD, ASR, cleanup — and why "it feels smooth" isn't the same as knowing it works.
#5459: Rebuilding the Podcast: Turn-Taking, Buttons, and Chatterbox
Daniel wants a push-to-interrupt button and a live voice loop. The research says the button is easy and the live part is a research project.
#5457: Pigeon Alerts on a Repurposed i5: Frigate Without the TPU
Daniel wants a pigeon notification, not a custom model. Frigate already ships with bird detection — the real work is masks, zones, and filters.
#5456: Parakeet vs Whisper: Picking a Phone ASR Model
Why Whisper loses on Android, why Parakeet v2 beat v3, and how to benchmark speech-to-text without any tooling.
#5447: The Models That Never Talk Back
Some models read text, score it, and return a number. No chat, no reasoning, just decisions — and they're running under every router you use.
#5435: When Your TTS Model Eats the Numbers
Numbers, dates, and acronyms break text-to-speech in specific, documented ways. Here's where normalization lives — and why it depends on your model.
#5430: Two Boxes: ASR and the Text Fixer Behind It
Punctuation, casing, ITN, disfluency — the four-job layer between raw ASR output and text you can actually read.
#5420: How Keyboard Middleware Actually Works
A keyboard isn't one thing — it's a stack. Here's how input methods, composition buffers, and hooks really work on Linux, Android, and Windows.
#5419: The Attention Budget in Your Pocket
Why phone dictation runs out of room, and how bounded attention windows buy you punctuation without blowing your memory budget.
#5411: Fine-Tuning at 4-Bit vs 16-Bit: What It Really Costs
QLoRA cuts fine-tuning VRAM 15x and cost up to 85% — but you pay in training time, quality, and safety alignment.
#5409: What Makes a Model "Agentic"? Atria Dawn Preview
A 744B-parameter model drops quietly on Hugging Face. Is "agentic" a real model category, or just a deployment pattern?