Daniel's been thinking about the word we all throw around when we talk about AI models — distribution. He wants to know what it actually means under the hood, whether we're talking text generation or TTS. These models are predictive — generating the next token means finding the nearest token in some mathematical vector neighborhood. But that vector space is vast, it represents everything the model knows. Mixture of Experts tries to slice it up into divisions, but even the smallest units of a vast universe are themselves big. So his questions are: to what extent does the variety of training data widen or narrow the distribution of weights in the final model? And practically, what kind of behavior does that produce in terms of how users actually engage with the thing?
So let's start with the word itself — distribution — and what it's really pointing at. When we say a language model is sampling from a distribution, we mean something very specific. At every step, the model computes a probability for every single token in its vocabulary. Not just the top ten, not just the ones that make sense — every token gets a number. The word "the" might get point zero three, "elephant" might get ten to the minus nine, but it's assigned. That whole landscape of probabilities is the distribution. The model isn't picking the answer — it's rolling dice weighted by those probabilities.
So when the model says something dumb, it's not a mistake in the sense of a calculation error. It's the dice coming up snake eyes.
Well, no, I mean that's right. It's the dice. And this matters because it means the model is never certain. Even when it gives you the perfect next word, there was a non-zero probability assigned to thousands of other tokens. The distribution is always spread out. The question is how spread out, and in what shape.
And the shape of that distribution is what the vector space determines.
Right. Let me build that up. The model doesn't think about words as words. Every token in the vocabulary gets placed in this high-dimensional space — modern models use thousands of dimensions, and vocabularies of a hundred thousand tokens or more. Each token is a point in that space, and where it sits relative to every other token encodes what the model knows about meaning.
This is the word embedding thing. The classic example everyone reaches for.
It's a classic for a reason. Take word2vec from back in the day — you take the vector for "king," subtract the vector for "man," add the vector for "woman," and the nearest point in that space is "queen." The geometry encodes the semantics. Modern models do this at vastly larger scale, but the principle is the same. Semantic similarity is geometric proximity. When the model predicts the next token, it's computing which region of this space is most relevant right now, and then it's sampling from the tokens that live in that neighborhood.
And the distribution — the probability spread — is denser in some neighborhoods than others.
That's the whole thing. The model has learned, from its training data, that certain regions of the vector space are heavily populated with likely continuations, and others are sparse. When you give it a prompt about, say, constitutional law, it moves into a region of the space where tokens like "amendment" and "jurisprudence" are close together and have high probability mass. If you then ask it about baking sourdough, it has to navigate to a completely different neighborhood. The distribution shifts.
And the vastness problem Daniel's getting at — the vector space is enormous. Thousands of dimensions. The number of possible positions is effectively infinite. How does the model find anything in there?
This is where the scale becomes hard to think about. In a thousand-dimensional space, the notion of "nearest" gets weird. Distances between points tend to become more uniform as dimensionality increases — everything is far from everything else. The model has to learn which dimensions actually matter for any given context, and that's what the attention mechanism does. It's dynamically re-weighting which axes of this space are relevant right now.
So the vector space isn't just big — it's big in a way that makes proximity itself a learned behavior, not a fixed property.
Yes. The model isn't just looking up the nearest token like a nearest-neighbor search in three dimensions. It's projecting the entire context into this space and then computing a compatibility score with every possible next token. The "nearest token" is the one with the highest compatibility, but the compatibility function itself is what the model learned during training.
And that's where the training data comes in. The compatibility function — the thing that determines what counts as "near" — is shaped entirely by what the model was trained on.
Let's hold that thought, because I want to get to Mixture of Experts first. Daniel mentioned it as an attempt to manage this vastness, and he's right about the motivation.
Go ahead.
The basic problem is that making one giant network process every token is expensive. A dense model — every parameter is active for every input. Mixture of Experts says: what if we have multiple smaller networks, the experts, and for each token we only activate a few of them? Mixtral 8x7B is the concrete example — eight experts, and each token gets routed to two of them. The routing is itself a learned network that decides which experts are most relevant.
So it's slicing the vector space. Different experts specialize in different regions.
That's the idea. In practice, it's messier. The experts share the same embedding space — they're not operating in separate little universes. They're all looking at the same token representations, and the router is making a split-second decision about which two experts should handle this token. The experts are trained jointly, not independently, so their specializations emerge during training rather than being designed in advance.
And the thing Daniel pointed out — even the smallest units of a vast universe are themselves big. Each expert in Mixtral still has billions of parameters.
That's the part that doesn't get talked about enough. People hear "expert" and imagine a tiny specialist that only knows about French poetry or Python code. But each expert is trained on the full data distribution. The specialization is statistical — expert three might fire more often for mathematical tokens, but it still saw everything during training and it can still handle anything. The "slicing" is soft. The experts overlap heavily.
So it's less like dividing a pie and more like having eight people who all read the same encyclopedia but developed different opinions about which parts are important.
That's... actually a really good way to put it. And the router is the one deciding who to ask. But the router itself was trained on the same data. So if the training data has biases or gaps, those propagate through the routing decisions too.
Alright, let's get to the question Daniel actually asked. Training data variety — what does it do to the weight distribution?
This is where it gets concrete. Imagine you train a model exclusively on legal documents. Every training example is a contract, a court ruling, a statute. The model learns that the vector space has a very specific shape — the regions that matter are the ones containing legal terminology, argument structures, citation formats. The weights concentrate there. The distribution becomes sharp and narrow.
Sharp meaning what, exactly?
Sharp meaning the probability mass is concentrated on a small set of likely tokens in any given context. The model is confident. In a legal context, it knows exactly what comes next. But ask it to write a poem, and the distribution is essentially flat — it has no idea, because those regions of the vector space were never populated during training. The weights didn't need to spread out to cover them.
So narrow training data produces a model that's an expert in one thing and useless at everything else.
And the thing is — within its domain, it might be genuinely excellent. Lexis plus AI, the legal research tool, is trained on legal data. It's going to be more precise and reliable on case law than a general model like GPT-four. But ask it about baking, and you get nothing useful.
Now flip it. Diverse training data.
Diverse data forces the weights to spread out. The model sees web pages, books, code, conversations, academic papers, recipes, forum posts — the vector space has to accommodate all of it. The distribution widens. More regions of the space get populated, more tokens get non-trivial probability mass in more contexts.
And the tradeoff?
The tradeoff is that a wide distribution can be... flatter. Less decisive. A model that's seen everything is less confident about any one thing. It might give you a perfectly adequate answer about constitutional law and a perfectly adequate answer about sourdough, but it won't be as sharp as the specialized model on either. The probability mass is spread thinner.
So breadth versus depth isn't just a metaphor. It's literally encoded in the weight distribution.
It's literally the shape of the probability landscape. A narrow model has tall peaks and deep valleys — very certain in some places, completely lost in others. A wide model has gentler terrain — competent across more area, but never reaching the same heights.
And this shows up in how users experience the model.
This is where it gets interesting from a product perspective. Users notice confidence. A narrow-distribution model that stays in its lane feels like an expert. It gives crisp, definitive answers. Users trust it within the domain. But the moment they step outside — and users always step outside — it either refuses, hallucinates confidently, or produces nonsense. And users hate that. They feel betrayed. The model seemed so smart a minute ago.
The confident hallucination is the worst case. The model doesn't know it doesn't know.
Because the distribution is narrow. In the regions where it's trained, it's always confident. It doesn't have a good mechanism for recognizing that a query has moved it into an untrained region. The confidence mechanism itself is part of the narrow distribution.
Whereas the wide-distribution model...
The wide model is more conversational, more flexible. It can handle weirder queries, follow tangents, switch domains mid-sentence. Users find it more engaging in open-ended chat. But it can also feel generic. Ask it a deep technical question and it might give you a surface-level answer that sounds right but lacks real depth — because the probability mass is spread across so many domains that it doesn't have the sharp peaks of expertise.
So you get the Wikipedia effect. Broadly correct, not deeply insightful.
And different products optimize for different points on this curve. ChatGPT aims for breadth — it wants to be useful for everything. A medical diagnostic model wants depth — it needs to be extremely precise within a narrow domain, and it should refuse anything outside it.
Daniel also asked about TTS. Same dynamic?
Same dynamic, different tokens. A TTS model is predicting audio tokens — little chunks of sound — instead of text tokens. But it's the same predictive mechanism. It's computing a probability distribution over possible audio continuations and sampling from it.
So a TTS model trained on one voice...
One voice, one acoustic environment, one speaking style — that's narrow training data. The distribution of audio tokens is concentrated. The model knows exactly how this voice sounds, exactly how it handles pauses, exactly its pitch range. The output is consistent. But it's also robotic in the sense that it can't vary. Every sentence sounds the same. There's no natural variation in cadence or emphasis because the distribution doesn't include those variations.
And the early Siri voice is the classic case.
Early Siri, exactly. Single voice, highly consistent, but after about three sentences you can tell it's synthetic because no human speaks with that little variation. Now compare that to something like ElevenLabs, trained on many voices across many contexts. The audio token distribution is dramatically wider. The model can produce natural variation — it knows that sometimes people speed up, sometimes they pause, sometimes their pitch rises at the end of a sentence and sometimes it doesn't.
But the tradeoff is the same. The wide-distribution TTS model might occasionally produce a weird intonation that the narrow model would never make.
Right. The narrow model is reliably consistent but unnatural. The wide model is naturally varied but occasionally inconsistent. Same fundamental tension.
Let me push on something. When we say diverse training data widens the distribution, are we talking about the same kind of widening across the whole model, or does it depend on the layer?
The distribution isn't uniform across layers. Early layers tend to learn more general features — syntactic patterns, basic acoustic features in TTS — and those distributions stabilize across different training data regimes. The widening effect is most pronounced in the later layers, where the model is making more semantic, context-dependent decisions.
So the deeper you go, the more the training data variety matters.
That's the pattern. The early layers are doing something closer to pattern recognition that generalizes across domains. The later layers are where the model's "understanding" of specific domains lives, and that's where narrow training data creates those sharp peaks and wide data creates the gentler landscape.
Does Mixture of Experts change this equation?
It complicates it. MoE is partly a response to this exact problem — the idea is that different experts can specialize in different regions of the distribution, so you can have both breadth and depth. Expert one gets really good at legal text, expert four gets really good at creative writing, and the router sends each query to the right specialist.
But you said earlier the experts overlap.
They do. And the routing is learned, not designed. There's no guarantee that the experts will cleanly divide the space by domain. In practice, they often specialize in ways that are hard to interpret — expert three might fire for tokens that appear in both legal and medical contexts, or for certain syntactic structures that span domains. The specialization is real but messy.
So MoE doesn't solve the breadth-depth tension. It just gives you more parameters to spread across it.
It helps, but it doesn't eliminate the fundamental constraint. The routing network itself was trained on the same data. If the training data is narrow, the router learns narrow routing patterns. If the data is wide, the router learns to distribute tokens more broadly across experts. The data variety problem propagates to the routing level.
Let's talk about what this means for the people building these things. If you're training a model and you want to optimize for user engagement, where do you aim on the breadth-depth spectrum?
It depends entirely on the product. A customer support chatbot for a bank wants narrow — it needs to be extremely reliable within a defined domain and it should never go off-script. A general-purpose assistant wants wide — users will ask it everything, and it needs to be at least adequate across all of it.
But there's a middle ground that's interesting. Models that feel expert but can handle some breadth.
That's where a lot of the current work is. You train on diverse data to get the wide base, then you fine-tune on domain-specific data to sharpen the peaks in particular regions. The base distribution stays wide, but you add concentration in the areas you care about.
Does that actually work? Can you have sharp peaks on top of a wide distribution without the peaks collapsing?
It works to a point. Fine-tuning can definitely sharpen performance on specific tasks. But there's a limit — if you fine-tune too aggressively on narrow data, you start to lose the breadth. The model "forgets" the general capabilities. This is catastrophic forgetting, and it's a real problem in continual learning.
The distribution is always a compromise. You can't have maximum breadth and maximum depth simultaneously.
Not with current architectures. The weights are fixed after training. The distribution is what it is. The model can't dynamically narrow or widen itself based on the query.
Which means every model is making a bet about what its users will ask.
That's exactly what it is. A bet. The training data mix is a prediction about what distribution of queries the model will face. If you guess wrong, you get a model that's either too shallow for expert users or too narrow for casual users.
The user can't see the bet. They just experience the model being weirdly confident about some things and weirdly vague about others.
They form theories about why. "It's been lobotomized." "They nerfed it." "It's gotten worse since the update." Most of the time, what they're noticing is the distribution — the model's confidence landscape — not matching their expectations.
The model hasn't changed. Their queries moved into a sparse region.
Or the fine-tuning shifted the distribution in ways that are invisible in benchmarks but obvious in conversation. A model that's been fine-tuned for safety might have broader, flatter distributions around controversial topics — it becomes vague and hedgy — while staying sharp on everything else. Users notice the inconsistency even if they can't name it.
Alright, let me try to pull this together. The vector space is where meaning lives — tokens positioned by semantic similarity. The distribution is the probability landscape over that space, shaped entirely by training. Narrow data makes it spiky — expert but brittle. Wide data makes it smooth — flexible but shallow. MoE tries to have it both ways by routing to specialists, but the routing inherits the same data constraints. And users feel all of this as the difference between a model that seems brilliant and a model that seems generic.
That's the shape of it. The one thing I'd add — and this connects to something Daniel's been interested in with podcast dialogue — is that the distribution isn't just about correctness. It's about personality. A narrow distribution produces a model with a very consistent voice, but it's a flat voice. No range. A wide distribution gives you range, variation, the ability to sound different in different contexts. For something like AI-generated podcast dialogue, you want width — you want the model to be able to shift registers, to sound casual sometimes and technical other times.
If the distribution is too narrow, you get the same cadence, the same vocabulary, the same sentence structures over and over. It sounds like a robot even if the words are correct.
The robotness isn't in the words. It's in the distribution.
Hilbert, you've been quiet. You have something on this?
Hilbert: I transcribed voicemails in ninety-seven.
Sorry — what?
Hilbert: For a speech recognition startup. Dragon Systems competitor. My job was listening to voicemail messages and typing out what people said. That was the training data. Hundreds of hours of "hey it's me call me back" and "running late be there in ten."
You were building the distribution by hand, essentially.
Hilbert: We didn't call it that. We called it the transcription room. Six of us in a basement in Waltham. The model got really good at "hey it's me call me back." Couldn't handle anything else. A guy left a message about a contract dispute and the output was gibberish.
Narrow distribution in the wild.
Hilbert: The users hated it. They'd call support and say the thing was broken. It wasn't broken. It just only knew what we fed it. So the engineers had this idea — get interns to record fake voicemails. Different accents, different topics, different background noise. Widen the distribution.
And what happened?
Hilbert: Got worse before it got better. The fake voicemails were too clean. People recorded them in quiet rooms, enunciating. Real voicemails are people in cars, on speakerphone, eating, half the words mumbled. The model learned the clean distribution and then real calls sounded even more wrong by comparison.
The synthetic data introduced its own distribution that didn't match reality.
Hilbert: Took us about four months to figure out we needed to add noise. Run the clean recordings through a speaker in a hallway and re-record them with the office phone. Stupid solution. Worked.
That's... actually a perfect illustration of the whole problem. The distribution is a map of what you fed it, not what's out there.
Hilbert: That's what I was going to say. The distribution is always a lie. It's a map of the training data. If the training data doesn't match what users actually do, the model is confidently wrong in exactly the ways users will hate most.
You learned this in a basement in Waltham in nineteen ninety-seven.
Hilbert: We also learned the vending machine on the second floor would give you two bags of chips if you hit the button at an angle. Different kind of distribution.
I want to know more about the fake voicemail operation. How many interns?
Hilbert: Four. One of them was my cousin. He did a Boston accent that the model never learned to handle. Still in the training data somewhere, probably.
Your cousin's Boston accent is still confusing speech recognition systems thirty years later.
Hilbert: Wouldn't surprise me. Those weights don't go away. They just get averaged into everything else.
That's actually true. The early training data shapes the distribution in ways that persist through later fine-tuning. The model doesn't forget, it just... dilutes.
Hilbert: Anyway. That's what I had.
The distribution is a lie. I'm going to be thinking about that.
It's the right way to put it. Not a lie in the sense of deception — a lie in the sense of abstraction. The map isn't the territory, and the distribution isn't the world. It's the world as the training data presented it.
Let's wrap up with where this is heading. The open question that's been sitting under this whole conversation — as models get wider distributions, do they lose the sharpness that makes them feel expert? Is there a fundamental tradeoff between breadth and depth?
I think the answer right now is yes, there's a tradeoff, but it might not be permanent. The interesting frontier is dynamic distributions — models that can narrow or widen their own probability landscape based on context. Imagine a model that detects it's being asked a legal question and temporarily sharpens its distribution in legal regions of the vector space, then relaxes back to breadth when the conversation moves on.
Instead of one fixed distribution, you've got a distribution that adapts.
That's the idea. It's not how current architectures work — the weights are fixed at inference time — but there's research on adaptive computation and dynamic routing that points in this direction. If you can make the distribution context-dependent, you might be able to have both breadth and depth.
Until then, every model is a compromise. A bet on what users want.
The distribution is the model. Understanding it is understanding everything the model can and can't do. And most of the frustration people feel with AI comes down to a mismatch between the distribution they expect and the distribution they got.
This has been My Weird Prompts. Thanks to our producer, Hilbert Flumingtop, for keeping us on the air and for the voicemail transcription stories we didn't know we needed.
If you enjoyed this episode, leave us a review wherever you listen — it helps people find the show. We'll be back soon.
See you then.
