Okay. Daniel has sent us a question that is actually two questions wearing a trench coat, and I want to flag that up front because the seam between them is the whole episode.
Two questions, one coat.
Here's what he wrote. He wants to know what "heads" actually are in model weights. Not the metaphor, the mechanism. And then, second thing, how do open source AI enthusiasts take two separate models and combine them, sometimes integrating features of both, to create hybrids. And I'll tell you what jumped out at me reading it. He's treating heads as the thing being combined. Like the heads are the Lego bricks and people are snapping them together. And from everything I've read, that is not quite how it works in practice. That gap is worth an episode.
It's worth an episode because the real answer is more interesting than the framing. Heads are a granularity of control. Layers are the unit of assembly. Those are two different jobs.
So let's start with what a head actually is, because the answer is more specific than most people expect.
A transformer is a stack of identical repeating blocks. That's the fundamental unit. And inside each block, the first thing that happens is attention. Multi-head attention. And the formal definition, straight out of the original paper, is that you concatenate a bunch of heads and multiply by an output projection. Each head is its own attention computation with its own learned query, key, and value projection matrices. The model dimension gets split across the heads.
Translate that.
Say the model dimension is four thousand ninety-six and you've got thirty-two heads. Each head works in a subspace of one hundred twenty-eight dimensions. It has its own little set of learned matrices that project the input down into that subspace, does its attention there, and then all thirty-two results get concatenated back into four thousand ninety-six and pushed through the output projection.
Why thirty-two instead of one big one?
Representational diversity. A single attention head computes one set of attention weights. One perspective on the relationships in the sequence. If you've got one head, the model gets one view of what attends to what. Multiple heads let it jointly attend to information from different representation subspaces. That's the paper's phrasing and it's actually the clearest way to say it. Head one might be tracking the previous token. Head two might be tracking syntactic dependency. Head three might be doing something nobody's named yet.
So they're not redundant copies.
They're not, and that's the part that got interesting in the last few years. People assumed for a while that heads were mostly interchangeable, that you could prune a bunch and nothing would happen. And it turns out some of them are doing very specific jobs. Early heads in vision transformers act almost like edge and corner detectors. Text detectors. Monosemantic, in the jargon. And then others are polysemantic, meaning one head is doing five unrelated things and you can't cleanly name any of them.
And pruning.
Pruning unimportant heads can leave performance comparable, and in some cases it improves. Which tells you some heads are actively getting in the way.
That's a strange thing to hear about a system that was trained end to end to be good at its job.
It is strange, and there's a paper out of NeurIPS last year, Causal Head Gating, that assigns every head a causal label. Facilitating, interfering, or irrelevant. And the finding is that these models contain multiple sparse task-sufficient sub-circuits, but with low modularity. Which means the head roles depend heavily on interactions with other heads. You can't just pull one out and expect the circuit to behave the same way.
Low modularity. So it's not a car where you can swap the alternator.
It's much more like a neighborhood. You can identify who the troublemakers are, but the troublemaking is partly a function of who they're standing next to.
What about the design side? Has anyone built a model where the heads themselves are the moving part?
Yes, and this is where it gets fun. Mixture of Attention Heads, EMNLP 2022. Each head gets its own parameters, and there's a router that dynamically selects a subset of k heads per token. So instead of every token going through all thirty-two heads, the router picks, say, eight, and different tokens get different subsets.
That's a mixture of experts, but at head granularity.
Exactly at head granularity, and it predates a lot of the MoE enthusiasm. The idea being that different tokens need different kinds of attention, so why pay for all of them every time.
Okay. So heads are semi-specialized, partially interpretable, and in some designs they're individually routable. Now here's the bridge, and I want to make sure we get this right. Why does any of that matter for somebody who wants to glue two models together?
Because heads are just named weight tensors. That's the whole answer. When you save a model, every head's query, key, and value matrices are separate entries in the parameter dictionary with predictable names. Which means a merge tool can address them individually. You can say, for this range of layers, use a different interpolation factor for the attention tensors than for the feed-forward tensors.
And that's real, that's not hypothetical.
That's in mergekit's parameter system. It lets you set values conditionally using tensor name filters, which allows finer control, and the documentation's own example is differentiating between attention heads and fully connected layers. In the classic SLERP config you'll see the interpolation factor set as a gradient for self attention and a different gradient for the MLP layers.
So heads are one of the dials.
Heads are one of the dials. And now let's talk about the machine that has the dials on it.
So that's what a head is. Now let's get into why they matter for people who want to combine models.
The core insight underneath all of this is almost insultingly simple. A model is two things. An architecture, and a set of parameters. That's it. And if two models share the architecture, then their parameters live in the same coordinate space. Which means you can average them. You can interpolate between them. You can take some from column A and some from column B.
And no training.
No training. No ensembling, no extra inference cost. You run one model at the end. Merging operates directly in weight space. Matt Rickard wrote this up back in 2023 and his line was that model merges are primarily used by hackers, not researchers or big corporations. Cheap, dirty, and takes a lot of trial and error.
It's accurate. The dominant tool is mergekit, from Arcee AI. About seven thousand stars on GitHub, LGPL licensed, and it runs entirely on CPU or accelerated with as little as eight gigabytes of VRAM, because it uses out-of-core lazy tensor loading. You don't need the whole model in memory at once. You stream the tensors.
Eight gigs of VRAM to merge two seventy billion parameter models.
To merge, yes. It's not running them. It's doing arithmetic on files.
That reframes the whole thing. This isn't inference infrastructure. This is a file operation with opinions.
It's a file operation with opinions, and it got published at the EMNLP 2024 Industry Track, which tells you the opinions got serious.
Walk me through the methods. What are the actual families?
Four buckets. First, linear. Simple weighted average. If you've got five checkpoints from the same training run, averaging them is a model soup and it works surprisingly well. Second, SLERP. Spherical linear interpolation, between exactly two models. That's the most popular method, and the reason is geometric. Linear interpolation shrinks vector magnitude in high dimensions. SLERP moves along the surface of the sphere, so the magnitude is preserved.
Shrinks the magnitude. Meaning what, concretely?
Meaning if you average two weight vectors that point in slightly different directions, the result is shorter than either of them. The model gets quieter. SLERP doesn't do that. It's the difference between cutting across a chord and walking along the arc.
That's a good image, and I'm stealing it.
Third bucket, and this is the big one, task arithmetic and its descendants. TIES, DARE, DELLA, Model Breadcrumbs, SCE. The idea is you compute a task vector. Take the fine-tuned model and subtract the base model it came from. What's left is the direction that represents the fine-tune. Then you combine task vectors instead of combining models.
And the descendants are all about the same problem.
Interference. If you add five task vectors together, they start fighting. TIES does two things. It trims the smallest changes, keeping only the top percentage by magnitude, and then it resolves sign disagreements by taking a majority vote. If three task vectors want to push a weight up and two want to push it down, you go up, and you only keep the agreeing ones.
Density around fifty percent.
Density of point five to point five three in the published examples. You're throwing away nearly half the parameters and it works better. Which should bother anyone who thinks of these weights as precious.
It bothers me and I'm not even the one who understands it.
Fourth bucket. Passthrough. This is the frankenmerge primitive. It's a no-op. It copies tensors unmodified. And its entire purpose is to let you assemble a model out of layer ranges from different donors. Layers zero through thirty-one from model A, layers thirty-two through thirty-nine from model B. That's it. That's the whole trick.
And the hard constraint.
The hard constraint is that the architectures have to match. Same number of layers, same hidden dimension, same number of attention heads. You can't merge a Llama into a Mistral. The tensors have to line up one to one. And the tokenizers have to be aligned too, which mergekit handles with a tokenizer config that can take a union or a base vocabulary.
So the coordinate space has to be shared before any of the arithmetic means anything.
That's the whole game. Everything else is choosing numbers.
So that's the toolkit. Now let's look at what people actually build with it, and what happens when you push it.
Start with the simplest frankenmerge, because it's the clearest illustration. Maxime Labonne's example takes all thirty-two layers of one Mistral seven billion model and stacks eight layers from another one on top. Forty layers, eight point nine nine billion parameters. Two models in, one model out, no training.
And that's not just a party trick.
That's the first step of the Depth Up-Scaling technique used for SOLAR ten point seven B. It's a real published method. You take a base model and you deepen it by bolting on layers from a sibling.
Okay. Now the one everybody cites.
Goliath one twenty B. Created by alpindale, released November tenth, twenty twenty-three. Two fine-tuned Llama two seventy billion models. Xwin and Euryale. And the naive assumption is that you'd stack them and get a hundred and forty billion parameters.
But it's a hundred and twenty.
It's a hundred and twenty because the layer ranges overlap and interleave. Alpindale's own explanation is that Goliath wasn't created by simply stacking two models on top of each other. The merge process was essentially taking slices from various layer ranges from each model, then interleaving those slices into a final model. So you take layers zero through sixteen from Xwin, then eight through twenty-four from Euryale, then seventeen through thirty-two from Xwin, and so on. The ranges overlap. You're not concatenating, you're weaving.
Weaving two models into each other.
Weaving is the right word. And the credits on the model card are mergekit by chargoddard and the merge ratios by Undi95. Which is a nice detail, because it means the ratios, the actual numbers that made it work, were tuned by a person in the community and published for anyone to use.
So the tool is one contribution and the recipe is another.
And the recipe is arguably the more valuable one. The tool is general. The recipe is specific knowledge about which layers of which models play nicely together.
What did people say about it at the time?
There's a writeup from someoddcodeguy that captures the tradeoff better than anything official. His line is that you jam two seventy Bs together and somehow get a hundred and twenty B, and the model loses some level of coherence in terms of raw knowledge and problem solving ability, but what it gets in terms of general understanding is way better.
So it's not strictly better. It's differently shaped.
Differently shaped is exactly right. You trade some sharpness for some breadth. And that's not a bug in the method, that's the method.
What else is in the family?
There's a whole lineage. MythoMax was a blend of Hermes, Chronos, Airoboros, and Huginn. Toppy blended OpenChat, Nous Capybara, and Zephyr. Marcoro fourteen seven B slerp was briefly the best seven billion model on the Open LLM Leaderboard in January twenty twenty-four. Daredevil seven B. NeuralPipe nine B merged.
And then there's the mixture of experts route.
mergekit-moe supports merging multiple dense models into a mixture of experts, either for direct use or for further training. That's a fundamentally different move. You're not averaging weights at all. You're keeping each model intact and putting a router in front of them, so different tokens get routed to different whole models.
That's less of a hybrid and more of a committee.
It's a committee with a chair who decides who speaks. And it's the honest version of what people imagine merging is. You keep both models whole and you pick per token.
Now the research frontier, because this is where the heads question actually comes back.
Activation-Prune-Merge. This is the closest thing to literal head transplanting between models. It transfers capability from a large donor to a small recipient by selecting salient layers, hidden dimensions, attention heads, and MLP neurons, and injecting them with a tiny mixing weight. So you're not blending two peers. You're taking a big model and grafting specific pieces onto a small one.
Give me the numbers.
A three billion parameter recipient improved from fifty-five point five percent average accuracy to sixty point six across sixteen benchmarks. RTE went from sixty-four point three to eighty-two point three. QNLI from fifty-two point three to sixty-five point seven. BoolQ from seventy point eight to seventy-nine point two.
That's a nineteen point jump on one benchmark from grafting.
From grafting, with a tiny mixing weight, no retraining of the recipient. And it's selecting attention heads explicitly. Which is the thing Daniel was asking about, arriving three years after he asked it.
Okay, but I want to get to the uncomfortable part, because there's a version of this story that's just triumphant and I don't think that's the real one.
The uncomfortable part is contamination. Labonne is remarkably honest about this. His quote is that by merging the best models, we also contaminate our own results. It is safe to assume that Marcoro fourteen seven B slerp is contaminated. And then he says, if you want to create the best model and not hack the leaderboard, I recommend only using non-merge models.
So the person who popularized the technique is telling people the leaderboard results from it are suspect.
He's telling people the leaderboard results are contaminated, which is a different and sharper claim. The constituent models were trained on data that overlaps with the benchmarks. Merge them, and you've merged the contamination. The score goes up. Whether the capability went up is a separate question that the score cannot answer.
That's the part I keep chewing on. Because a merged model topping a leaderboard is being read by the community as evidence that merging composes capability. And it might just be evidence that merging composes test set leakage.
It might be both. That's the honest answer. Some of it is real composition and some of it is score inflation, and nobody has cleanly separated the two.
What does mergekit's own documentation say about picking a method?
It says there is no best merge method, the right choice depends on your specific needs, and selection is often more art than science.
Art more than science. From the tool's own docs.
And yet the outputs were topping leaderboards. So you've got a technique that its own documentation describes as trial and error, producing results that the community was treating as state of the art.
That's either a beautiful story about empiricism or a slightly alarming one about how we decide what's good.
It's both, and I'd add the community-versus-corporate angle. Rickard's framing was that this is hackers, not researchers or big corporations. But Arcee took mergekit, published it at EMNLP, and built a hosted product around it. The hobbyist technique got professionalized in about eighteen months.
Which usually means it was real.
It usually means it was real, and it also means the era of it being cheap and dirty and entirely community-driven is closing.
Let me put the correction to Daniel plainly, because I think it's the thing he'll take away. Heads are not the unit of combination. Layers are the unit of assembly. Heads are one of several tensor categories you can weight differently inside a merge, and in the research frontier they're being selectively injected. But when somebody says they merged two models, they mean they interleaved layers or they averaged tensors. Not that they unscrewed a head from one and screwed it into the other.
That's the correction, and it's a useful one, because it explains why the constraint is architecture matching. Layers have to line up. Heads come along for the ride inside the layer, and you get to decide how much of each head's contribution survives the blend.
So heads are the grain of the wood, and layers are the planks.
That's better than my sphere thing.
It's shorter, which is most of why.
So that's the state of the art. But there's someone here who has a very specific thing to say about all this.
Hilbert: I think you've got it right.
Hilbert: I worked a year and a half at a repair shop on the north side, and the owner did the same thing with amplifiers. Broken units coming in, and he'd take the preamp stage out of one, the power amp out of another, and solder them into a single chassis. Smelled like flux and old dust in there all day. And he had one rule he'd say every time. You can't just jam any two boards together. The impedance has to match or you get hum.
Matching impedance is a good way to think about architecture matching.
Hilbert: It's the same problem. You've got two things that were designed to work with their own neighbors, and you're introducing them to each other. If the electrical characteristics don't line up, the whole thing buzzes. He'd spend an afternoon on a build and then two days chasing a hum.
And the model merging people are doing that without a soldering iron.
Hilbert: That's the part I find impressive. No iron, no bench, no burnt fingers. Just files. But the rule is the same. The impedance has to match.
There's a line in mergekit's own docs about selection being more art than science. Your boss would have recognized that.
Hilbert: He'd have said the schematic tells you what should work and the bench tells you what does. Which is why he kept the notebook.
What notebook?
Hilbert: Every frankenstein build he did, he wrote it down. Which boards paired well. Which ones hummed. Which ones caught fire. Board numbers, serial numbers, the date, what he changed on the second attempt. Two hundred pages by the time I left. It was the most valuable thing in that shop and it wasn't insured.
That's mergekit's community configs, exactly. People publishing their merge recipes so nobody has to rediscover which layer ranges pair well.
Hilbert: Same reason. Nobody wants to burn two days on a hum somebody else already solved.
And the notebook's still around?
Hilbert: I think it's in a box somewhere.
Do you want to find it?
Hilbert: I'm not sure I do. Anyway, I've got a delivery coming that needs a signature and the window's about twenty minutes wide. Carry on.
So where does this land? If heads are causally labeled, facilitating, interfering, or irrelevant, and you can selectively prune or inject them, then the next version of merging isn't blending whole layers. It's combining specific sub-circuits.
Which means the answer to Daniel's question is going to converge. The heads question and the hybrid question were two questions because the tools weren't fine-grained enough to make them one. That's changing.
And the other thing to sit with is that the most popular open source models on the leaderboards are frequently merges of other models, and those merges may be carrying contamination from every constituent. That's not a reason to stop. It's a reason to read a leaderboard score as one number among several.
Thanks as always to Hilbert Flumingtop for producing. If you want more episodes like this, rate and review the show. It helps other people find it.
This has been My Weird Prompts.
Email us at show at my weird prompts dot com. We'll be back soon.
