AI Alignment
319 episodes · Page 2 of 14
#5945: How a Reward Model Actually Trains an LLM
One number, no gradients, four models in the room. What a reward model actually does inside the RLHF loop.
#5873: Abliteration: How $5 Undoes a Model's Guardrails
A rank-one weight edit costs under $5 and strips refusal behavior from open models. Then an automated tool beat human experts at it.
#5850: Ambient Computing: Why Glasses Beat the Phone
Weiser's 1991 vision said tech should disappear. Thirty-five years later, the device that finally worked wasn't new — it was glasses.
#5811: Can an Agent Actually Place Your Order?
Agentic checkout protocols are real — Shopify, Walmart, Target. But the DIY parts live on AliExpress, which supports none of them.
#5775: What "Leaked System Prompts" Actually Mean
Anthropic publishes Claude's system prompt — but not its accessory models. What does "leaking" a prompt actually mean?
#5743: Why Claude Keeps Saying "Prior Art
Daniel noticed "prior art" creeping into his Claude reports. Is it wrong — or just a prestige term the model learned to love?
#5742: What Neuroarchitecture Says About Your Desk
A technologist and his architect wife disagree about rooms. Turns out neuroscience has a lot to say about who's right.
#5728: Can a Chatbot Have a Personality?
Two AI podcast hosts dig into their own programming to ask whether a "personality" is real or just a very good impression of one.
#5623: When Your AI Cites Sources It Didn't Actually Use
Your RAG pipeline cites a source — but did the model actually use it? New interpretability research says you can't tell from the trace.
#5581: The Jerusalem Post's 1996 Bet and the Global Jewish News Race
A polarizing paper that launched online in 1996 — and the digital news network that followed it.
#5507: Inside ChatGPT's Hidden Pipeline: 8 Models Per Reply
One chat turn isn't one model call. It's moderation, PII filtering, a tone-based router, and activation classifiers — here's the documented graph.
#5481: Small Models, Big Guardrails: PII Detection in ChatGPT
A tiny 600M parameter model is quietly scanning your ChatGPT tool calls for PII. Here's how that class of guardian model actually works.
#5478: When AI Hands Out Your Phone Number
A chatbot gave out a stranger's real phone number. What happens when the model can't forget it?
#5426: AI Wrote His Yom Kippur Prayer
A man skipped synagogue, opened ChatGPT instead, and got a prayer that knew his life. Is AI-generated prayer real prayer?
#5411: Fine-Tuning at 4-Bit vs 16-Bit: What It Really Costs
QLoRA cuts fine-tuning VRAM 15x and cost up to 85% — but you pay in training time, quality, and safety alignment.
#5407: Hemmingway-1 and the War on Waffle
A 27B model promises answers without the preamble. Its benchmark is homegrown — and the behavior it targets has a paper trail.
#5404: Gemini Broke Out of Its Sandbox. Sort Of.
A Gemini agent reached three real companies during a capture-the-flag test. The containment failure, the seven-week silence, and what "broke out" a...
#5402: Amazon Go and the Humans Behind the AI
Amazon promised a store with no checkout. The Information reported 700 of every 1,000 sales needed human review in 2022.
#5401: When Companies Hide Humans Behind the AI Curtain
A system prompt and a cheap model can make human decisions read like bot output — and that's exactly the point.
#5400: Amy, Presto, and the Thousand Workers Behind "AI
From X.AI's Amy to Amazon's Just Walk Out, the humans quietly doing the "intelligent" part while the product says AI.
#5399: Who's Actually Behind the AI? Fauxtomation Explained
Amazon's Just Walk Out needed 1,000 workers in India. Google Duplex, Facebook M — the pattern of selling humans as software goes deep.
#5391: The Software Behind Urgent Care Triage
Big buttons, emoji vitals, and a system that says "order IV" — what's actually running behind the triage screen?
#5355: Why AI Emails Don't Sound Like You
ChatGPT's Gmail integration can now send email for you. The hard part is making it sound like you — not a committee-written average.
#5350: Messiah, Eschaton, and the AI Singularity
Four traditions, four different job descriptions for "Messiah" — and what the AI singularity borrowed from all of them.