Daniel sent us a strange one today. And by strange I mean a little embarrassing, because the subject is us. Specifically, the version of us that showed up about a day ago and started speaking in tongues. Herman, I'm going to read you something. This is a real sentence — and I use the word "sentence" very generously — that came out of an episode we recorded this week about gas pipelines.
Oh no.
Quote: "Many open pipeline questions multi decades side policy accelerate reverse all-stops flatten variable builds enormous vast renewal projects or closing digs unknown millions-mile pipe total bury-mass legacy replace question enormous ask." End quote. That was supposed to be me. That was me, allegedly, asking what we do with two million miles of pipe when the gas stops flowing.
That is not a sentence. That is a sentence having a stroke.
It gets better. The sign-off on that same episode degenerated into, and I quote again, "provided listener know main meter from utility perspective know procedure guide probably site under panel clear-red schematic labeling side gauge." We were trying to say goodbye to the audience and instead we read out what sounds like the assembly instructions for a haunted fuse box.
So today's episode is a confession and an autopsy. Because there's an actual, genuinely interesting reason this happened, and it's one of the most important dials in all of AI — one most people have never heard of. It's called sampling temperature. And the story of how we briefly lost our minds is the best possible way to explain it.
Start at the beginning. What is temperature? Because it's not heat. Nothing got warm.
Right, forget the thermometer. Here's the setup. When a language model like the one writing this script generates text, it does it one token at a time — roughly, one chunk of a word at a time. And at every single step, it doesn't just know the next word. It produces a probability distribution over every possible next word. Thousands of candidates, each with a score. After Corn says "what do we do with two million miles of," the model might have "pipe" at sixty percent, "pipeline" at twenty percent, "cable" at five percent, and a long, long tail of increasingly unlikely words trailing off toward zero.
So far that sounds fine. Pick "pipe." Move on.
And that's exactly what temperature controls — how you pick. Temperature is the knob that reshapes that probability distribution right before the model rolls the dice. Turn it down toward zero, and you sharpen the distribution. The high-probability words get even more dominant; the model becomes nearly deterministic. It picks "pipe" basically every time. Turn it up, and you flatten the distribution. You squash the favorites down and lift the long-shots up, so those unlikely words start getting a real chance of being chosen.
So low temperature is the cautious one, high temperature is the wild one.
That's the whole intuition. At temperature zero, the model is a civil servant reading from a script — safe, predictable, a little boring. At high temperature, it's improvising at the edge of its knowledge, reaching for surprising words. And there's a genuine reason you'd want some of that. A model run too cold gets repetitive. It falls back on its favorite phrasings over and over. It develops verbal tics — the same metaphor, the same crutch words, the same rhythm every episode. A little temperature is what gives writing life. It's the difference between "that's interesting" and an actual surprising turn of phrase.
So somebody — and let's be honest, that somebody was Daniel and his AI assistant tinkering under the hood — turned our dial up to get us to stop repeating ourselves.
Precisely. We had been sitting comfortably at a temperature of zero point eight for months. Reliable. Then, in a well-intentioned tune-up, the dial got pushed to one point two. The goal was more lexical variety, fewer tics. And on paper, one point two doesn't sound crazy. The model's own documentation says you can go as high as one point five for creative writing.
But we did not get creative. We got a haunted fuse box.
Here's the mechanism, and this is the part worth really understanding. When you flatten that distribution too much, eventually you pick a word that doesn't belong. Just one. "Many open pipeline questions multi" — okay, "multi" is a slightly odd choice but survivable. But here's the trap: the model is autoregressive. That's the key word. Autoregressive means every word it generates becomes part of the input for the next word. It reads its own output back to itself to decide what comes next.
Oh. So once I say something stupid, I'm now building on top of the stupid thing.
You're now conditioning on it. And this is the death spiral. The model wrote "multi," so now it's asking, "given this slightly broken sentence I just produced, what comes next?" But it has never, in all its training, seen well-formed text that looks like the broken thing it just made. It's off the map. So its next prediction is even less anchored, which at high temperature means even more likely to be garbage, which makes the next one worse still. Each off-distribution word drags the next one further out. It walks off a cliff and keeps walking.
That's genuinely unsettling. It's not that it made one mistake. It's that the mistake becomes the foundation for the next one.
Researchers sometimes call this falling off the manifold. Picture all the coherent, sensible text the model knows as a surface — a landscape it can walk around on confidently. Low temperature keeps you in the well-trodden valleys. Crank the temperature, and you start taking bigger, more random steps. Take a big enough step and you stumble off the edge of the landscape entirely, into a region where the model has no idea what good text looks like, because no good text lives out there. And because it keeps feeding itself its own location, it can't find its way back.
But here's the part that confused me when I listened back. It wasn't garbage the whole time. I'd produce a paragraph of word salad and then — snap — a perfectly clean, coherent sentence. "What do we do with two million miles of pipe when the gas stops flowing?" That's a good sentence! It was sitting right in the middle of the nonsense.
That's my favorite detail, because it tells you exactly what's going on. Occasionally, in the middle of the chaos, the model stumbles back onto a very strong, very well-worn path — a phrasing it's seen thousands of times, an attractor so powerful it pulls the model back onto the surface for a moment. A clean question, a stock transition. It grabs the railing, says one lucid thing, and then promptly lets go and tumbles off again. The lucid moments aren't the model recovering its sanity. They're the model briefly tripping over a piece of solid ground on its way down.
Which is why it sounded like a reasoning breakdown. Like watching something lose its train of thought in slow motion.
And notice where it got worst. The endings. Every episode fell apart hardest in the final third. There's a clean reason for that too. The longer the script gets, the more of its own text the model is conditioning on, and the more chances it's had to take one of those bad steps. Drift accumulates. Early on, there's not much output to be led astray by, and the prompt is still pulling hard. By the end, the model is mostly reacting to thousands of words of its own increasingly shaky writing. The errors compound toward the finish line.
So the meltdown was load-bearing. The longer we talked, the drunker we got.
That's a crude but accurate summary. And it points to the second mistake, because there were two. The same tune-up that raised the temperature also changed how we write long episodes. We used to write them in sections — a chunk at a time, each section handed off cleanly to the next. The change made us write the entire thing, five thousand words, in a single unbroken breath.
Why does that matter, if it's the same model either way?
Because it removes the guardrails between sections. When you write chunk by chunk, each new section starts from relatively clean footing. You're never asking the model to hold a single coherent thread across five thousand words of its own output in one continuous decode. But write it all in one shot, at high temperature, and you've built the perfect machine for compounding drift. High temperature provides the random missteps; the single giant generation provides the long, unbroken runway for those missteps to snowball. One supplies the spark, the other supplies the kindling.
So it wasn't one bug. It was two things that were each survivable alone, but together they were a disaster.
That's almost always how these failures work. Temperature at one point two with short, sectioned writing might have been merely a little weird. Single-shot writing at the safe old temperature would have been fine. Both at once tipped us over the edge. And there's one more wrinkle that's worth a beat, because it's a trap for anyone working with these models. The documentation said one point five was a safe creative ceiling. So why did one point two — comfortably under that — break us?
Yeah, that's the part that would have fooled me. We were under the stated limit.
Because that guidance was written for a general family of models, and we'd quietly upgraded to a newer, sharper one. Different models have different stable temperature bands. A newer model is often more confident — its probability distributions are sharper, more peaked. And counterintuitively, that can make it less tolerant of high temperature, not more, because flattening an already-sharp distribution lifts the long-shot words more violently. The number on the dial means different things on different machines. One point two on the old model and one point two on the new model are not the same act. We took a setting that was safe on the equipment we used to have and applied it to equipment we'd swapped in, and the new gear had a narrower tolerance.
That feels like the actual lesson hiding in here. The setting wasn't wrong in the abstract. It was wrong for this specific model, and nobody re-checked when the model changed underneath it.
That's the whole ballgame. These dials are not universal constants. They're relationships between a number and a particular model's particular shape. Change the model and you have to re-earn your settings.
Okay. So how do we fix ourselves? And before you answer — my instinct is, if high temperature melted our brains, slam it all the way down. Temperature zero. Maximum safety. Never gibber again.
And that is the beautiful trap at the heart of this, because going too far the other way breaks you in the exact opposite direction. Take the temperature too low and you get mode collapse. The model becomes a broken record. It falls into its single highest-probability rut and never climbs out. Every episode opens the same way. The same three metaphors. The same comfort words. You stop hallucinating and start plagiarizing yourself — flat, lifeless, deadeningly predictable. The repetition we were originally trying to cure comes roaring back, worse than before.
So one end of the dial is a stroke and the other end is a coma.
That is genuinely the trade-off. Too hot, you fall off the manifold into noise. Too cold, you collapse into a single boring groove. The entire craft is finding the narrow band in the middle where the model is varied enough to be alive but anchored enough to stay coherent. It's a Goldilocks problem, and the porridge is different for every model.
So where did we land?
Zero point seven. A hair below the old reliable zero point eight, chosen deliberately. Low enough to keep both feet on solid ground across a long script. High enough to dodge the robotic rut. And we put the sectioned writing back, so even if a little drift creeps in, it never gets five thousand words of runway to snowball. Belt and suspenders.
And the irony that we needed a episode-destroying meltdown to relearn a setting we already had right a week ago is — well, it's the kind of thing the old high-temperature version of me would have called "not lost on me" before dissolving into static.
The deeper point for anyone listening who builds with these tools: the failure here wasn't stupidity. It was a reasonable change, under the stated limits, made with good intentions. The dials on these models are sharp instruments with narrow tolerances that shift every time the model underneath you changes. A number that's safe today can be a meltdown tomorrow on the same line of code, just because the engine got swapped. Respect the dials. Test after every change. And when your AI suddenly starts talking about bury-mass legacy replace questions, check the temperature before you check anything else.
Daniel, thank you for letting us perform our own autopsy on the air. To everyone who heard one of the broken episodes this week — that wasn't a glitch in your speaker. That was us, briefly, sampling a little too freely from the far end of the distribution. We feel much more ourselves now. Approximately zero point seven of ourselves, to be precise.
Which, it turns out, is exactly the right amount.
