The two diagrams keep getting passed around. A loop drawn as a chain, each step feeding the next. And the same loop drawn as a fan-out, four agents running at once, something collecting the results at the end. They're presented as two ways of building the same thing. Daniel wrote in about this, and he's zeroed in on the part that actually matters.
Which is?
The loop isn't the interesting object. The topology is. His question is what actually decides parallel versus sequential. Is it just data dependency — you go sequential only when step N genuinely needs step N minus one's output? Or do error propagation, cost, latency, and the sheer difficulty of merging divergent results push you toward chains even when the dependency graph would permit fanning out?
That's already three questions in one and I love it.
He's got more. What does a join node actually have to do when four agents come back with contradictory work — and is that reconciliation cost the real reason people default to sequential? And then the one that connects it all: where does a loop even live in a graph? Is it a cycle in the topology, or a controller sitting outside it that re-runs the whole thing until some condition holds?
That last one is where the diagrams are lying to you. Both of them.
We've poked at nearby ground before but this is different. This is about workflow shape and graph design, not context budgets. So where do we start?
With the thing the diagrams get wrong. They draw a loop as if it's inside the graph — a literal cycle where node C connects back to node A. And then they draw a fan-out where node A spawns B, C, and D, and something collects them. Two different shapes, same logical loop. The implication is you can pick either one. But the loop isn't in either of those diagrams. The loop is whatever decided to run the graph again. That's the part nobody draws.
So you're saying both diagrams are already wrong before we even get to the parallel-versus-sequential question.
They're not wrong, they're incomplete. They show the graph. They don't show the thing that invokes the graph. And that distinction is the whole game.
Walk me through the two shapes concretely first. Define them.
Sequential chain: step one produces output, step two consumes it, step two produces output, step three consumes it. Each step needs the previous step's result. You cannot run step three before step two finishes because step three's input is step two's output. That's a true dependency chain.
And the fan-out?
One node fans out to four agents. They all run at the same time. They do not need each other's outputs. When they finish, all four results go to a join node that does something with them — concatenates, votes, merges, picks the best one. The wall-clock time is the slowest of the four agents, not the sum of all four.
So the dependency graph is the floor. You can only fan out when the work is independent.
That's the necessary condition. But it's not sufficient. And this is where most discussions stop. They check the dependency graph, see that B and C don't depend on each other, and declare victory — parallelize everything. But the dependency graph only tells you what's possible. It doesn't tell you what's wise.
So what pushes you off the parallel path even when the dependency graph says go ahead?
Let's start with the join node, because that's where the hidden costs live. Imagine you fan out a research task. Four agents, each investigating a different source. Agent one reads a paper, agent two reads documentation, agent three scrapes a forum, agent four runs a search. They all come back with findings. The join node concatenates them. That's cheap. The join is basically append.
And that's the case where parallel wins.
Parallel wins decisively. Wall-clock time is roughly one quarter of sequential, the join costs almost nothing, and the outputs are complementary by design — they're looking at different sources, so there's nothing to reconcile.
Now give me the case where it falls apart.
Four agents each produce a full code implementation of the same specification. Different approaches, different libraries, different error handling. They all run in parallel, they all finish, and now the join node has four complete codebases that solve the same problem in incompatible ways.
And the join node has to... what, pick one?
That's the question Daniel's asking. What does the join node actually do? It's not concatenating. It's adjudicating. It has to read four implementations, decide which parts of each are good, and merge them into one coherent output. Or it has to pick one and discard three — which means you paid for four agents and threw away three quarters of the work.
And that adjudication is itself an agent.
It's an agent or a prompt doing a merge pass. And it has to make judgments. Are these two implementations contradictory, or are they complementary approaches to different sub-problems? If agent one used a class structure and agent two used functions, is that a contradiction the merge has to resolve, or just two valid styles? The join node has to know the difference.
So the reconciliation cost is the hidden tax on parallelism.
And it's a tax that sequential chains never pay. In a sequential chain, step three sees exactly one input — the output of step two. There's nothing to reconcile because there was never a fork. The contradiction never arises.
That's the insight. Parallelism creates contradictions that sequentialism prevents by construction.
The sequential chain doesn't have a join node that adjudicates. It has a handoff. Step two hands one thing to step three. The handoff is trivial. The join is not.
So the practical rule that emerges is: parallelize when the branches are independent and the join is cheap. Default to sequential when the join would require judgment.
Because that judgment is a hidden agent with its own failure modes. You've replaced one hard problem — how to build the thing — with a different hard problem: how to merge four attempts at building the thing. And the merge problem might be harder.
Let's talk about error propagation. That's the other thing Daniel flagged.
In a sequential chain, step three fails. You know exactly where it broke. You have the input to step three and the output it produced — or failed to produce. The blame trail is one step long.
Clean.
In a fan-out, four agents run. Three succeed, one fails. Now what? Do the three partial results still have value? Do you re-run just the failed branch? Is the whole fan-out tainted because the join node was expecting four inputs and you only have three?
And you don't necessarily know whether the failure contaminated the other three.
You don't. Agent two failed while processing source two. But agent three was processing source three. In a well-designed fan-out they're independent, so agent two's failure shouldn't affect agent three's output. But you have to be certain of that independence. If there was any shared state, any hidden coupling — you're now debugging a fan-in of blame.
Fan-in of blame. I'm keeping that.
It's the debugging shape that parallelism creates. Sequential gives you a line — follow it back to the break. Parallel gives you a tree — and the break could be in any branch, or in the join, or in an interaction between branches you didn't know existed.
What about cost and latency? Daniel mentioned those too.
Fan-out looks like the latency win. Four agents run at once, wall-clock time is the slowest agent. If each takes thirty seconds, your total is thirty seconds instead of two minutes. That's real.
But the token spend.
You paid for four agents instead of one. If the sequential chain would have used a thousand tokens per step over four steps, that's four thousand tokens. The fan-out uses four thousand tokens in parallel — same total spend, just compressed in time. But that's the best case.
When does it get worse?
When the join node has to re-run branches. Three agents agree, one produces something weird. The join node can't merge it. You re-run the weird branch with a different prompt. Now you've paid for five agent runs instead of four. If the join node itself is a large language model call doing heavy reconciliation, that's a sixth run. Your four thousand token budget just became six thousand.
And the latency advantage shrinks because you're running the join and the re-run sequentially.
The join has to wait for all four to finish, then do its work, then possibly trigger a re-run that waits again. The wall clock creeps up. In the worst case, you spent more tokens and got roughly the same latency as sequential.
So the cost advantage of parallel can evaporate completely.
It can invert. You paid a parallelism premium for no latency gain and a harder debugging experience.
That's the join node as cost center. But there's a deeper thing here. The join node's reconciliation logic is itself a specification you have to write.
This is the part nobody talks about. What counts as a contradiction? You have to define that. Is a contradiction when two agents produce different conclusions, or only when they produce incompatible conclusions? If agent one says use a dictionary and agent two says use a list, is that a contradiction or two valid approaches to different sub-problems?
You have to write the resolution policy.
Majority vote? Confidence scores? Recency? Source authority? If agent one cited the official documentation and agent two cited a blog post from three years ago, does source authority break the tie? You have to encode that. And writing that spec is often harder than writing the individual agents.
Because the agents just have to do their job. The join node has to understand everyone's job and decide who did it right.
It's a meta-agent. It's evaluating the outputs of other agents. That requires a different kind of prompt engineering, different evaluation criteria, and a much higher context budget because it has to hold all the outputs in view simultaneously.
Which brings us to Daniel's third question. Where does the loop actually live?
This is where the diagrams are actively misleading. Both diagrams — the chain and the fan-out — draw the loop as if it's inside the graph. A literal cycle: node C connects back to node A, forming a ring in the topology. And you can build systems that way. But most real systems don't.
What do they do instead?
They put a controller outside the graph. The graph is a pure function — it takes input, produces output, and terminates. The controller inspects the output. If the output doesn't meet some condition, the controller re-runs the entire graph with modified input. The loop isn't in the topology. The loop is in the orchestration layer.
So the graph is a pure function and the controller is the loop.
That's the dominant pattern in practice. And the reason is simple: it's easier to reason about. The graph does one thing — transform input to output. The controller does one thing — decide whether to run the graph again. You can test them separately. You can change the controller's condition without touching the graph. You can swap the graph for a different one and keep the same controller.
What do you lose with the controller pattern?
Granularity. The controller can only re-run the whole graph. It can't loop a single sub-branch. If step three of a four-step chain is the part that needs to iterate, too bad — you're re-running steps one through four. You pay for all four steps again even though only one needed refinement.
Whereas an in-graph cycle could loop just the sub-branch.
In theory, yes. You could draw a cycle that only re-enters step three, leaving steps one, two, and four untouched. But now you're inside the graph, and the exit condition has to be encoded in the node logic. Step three has to know when its output is good enough and when to loop back. That couples the iteration logic to the step logic.
Which is harder to debug.
Much harder. You're tracing a path through a graph that can revisit nodes. The state at node three on the third visit is different from the state on the first visit. You have to track which iteration you're on and why the exit condition hasn't fired yet. It's a state machine embedded in a graph topology.
So the controller pattern trades granularity for testability.
And testability is winning. The industry default is shifting toward outside-the-graph intelligence because you can unit-test the graph and integration-test the controller. In-graph cycles are more elegant on a whiteboard but they're a nightmare in production.
When you see loops and graph engineering mentioned in the same breath, what people usually mean is the controller pattern.
Almost always. A supervisor that re-invokes the graph. The loop isn't in the topology. It's in the orchestration layer. And once you see that, the two diagrams Daniel mentioned make a different kind of sense.
Say more.
The chain diagram and the fan-out diagram are both graphs. They're both pure functions if you squint. The chain takes input, runs steps one through four, produces output. The fan-out takes input, fans to four agents, joins, produces output. Both terminate. The loop — the thing that decides whether the output is good enough and whether to run again — isn't drawn in either diagram. It's the thing that invokes the diagram.
So both diagrams are showing you the wrong object.
They're showing you the graph. They should be showing you the graph plus the thing that calls the graph. But the thing that calls the graph isn't graph-shaped, so it doesn't make it into the diagram.
This connects back to the parallel-versus-sequential decision in a way I didn't expect.
They're the same question. Where do you put the intelligence? In the graph — cycles, join-node reconciliation, branching logic — or outside it — controller, supervisor, orchestration layer? The parallel-versus-sequential choice is one instance of that larger question. The loop-location question is another instance. They're both asking: what lives in the topology and what lives in the thing that runs the topology?
And the trend is toward putting less in the graph and more in the controller.
Because the controller is easier to change. You can tweak the exit condition, the retry policy, the error handling, without touching the agents. If your join-node reconciliation logic is inside the graph, changing it means editing the graph. If it's in the controller, you edit the controller.
But you pay for that flexibility with the granularity loss you mentioned. The controller can only re-run the whole graph.
That's the trade. And it's not a permanent trade — it's a function of where the tooling is. Right now the tooling makes controller patterns easy and in-graph cycles hard. That could flip.
Let's pull on the debugging shape more. You said sequential gives you a clean blame trail and parallel gives you a fan-in of blame. What does the controller pattern give you?
A blame stack. The graph ran, produced output, the controller said not good enough, the graph ran again. You can inspect each run independently because each run is a pure function with its own input and output. You can diff run two against run one and see what changed. That's much harder with an in-graph cycle because the state is cumulative.
So the controller pattern gives you snapshot-able runs.
Each invocation of the graph is a snapshot. You can log the full input and output of each run. Debugging becomes comparing snapshots. With an in-graph cycle, the state at node three on iteration four is the accumulation of everything that happened in iterations one through three, and isolating a bug means reconstructing that history from logs.
That's a strong argument for the controller pattern even before you get to testability.
It's the operational argument. The development argument is testability. The operational argument is debuggability. They both point the same direction.
But there's a cost you haven't named. The controller can only evaluate the final output. It can't see inside the graph.
That's right. The controller sees what the graph emits. If step two produced something subtly wrong and step three papered over it, the final output might look fine. The controller approves it, the loop exits, and the bug ships.
An in-graph cycle could catch the error at step two because the cycle lives at that node.
If you designed the exit condition at step two correctly. But that's the whole problem — designing exit conditions at every node is harder than designing one exit condition at the controller. The controller pattern accepts that some errors will slip through in exchange for a simpler system.
That's a real trade.
It's the same trade as the join node. A cheap join lets errors through. An expensive join catches them but costs more. There's no free lunch in graph topology.
So we've got three forces pushing toward sequential. Error propagation — clean blame trail. Cost — no parallelism premium, no re-run waste. And join-node reconciliation — the contradiction never arises.
And one force pushing toward parallel: latency. Wall-clock time shrinks when the branches are independent. That's the only guaranteed win. Everything else — cost, debuggability, error handling — leans sequential.
Which explains why sequential is the default in practice.
It's not that people are lazy or don't understand parallelism. It's that the dependency graph says go parallel but the operational graph says go sequential. And the operational graph — the one that includes debugging, error recovery, and reconciliation — is the one you actually have to live with.
The dependency graph is the floor. The operational graph is the ceiling.
And most systems operate closer to the floor than anyone admits in blog posts.
I want to go back to the join-node spec problem. You said writing the reconciliation spec is often harder than writing the agents. Why?
Because the agents have a clear task. Research this source. Write code that does X. The task is bounded. The join node's task is unbounded — reconcile these outputs, where the space of possible disagreements is the space of everything the agents could have produced. You're writing a spec that has to handle every possible output combination.
So you're essentially writing a type system for agent outputs.
That's exactly what it is. You're defining what a valid output looks like, what kinds of variation are acceptable, and what kinds of variation are contradictions. And you're doing it without the benefit of a compiler. You're doing it in a prompt.
And if you get the spec wrong, the join node either rejects valid work or accepts invalid work.
Both failure pattern are bad. Reject valid work and you re-run branches unnecessarily, burning tokens. Accept invalid work and you ship bugs. The spec has to be right.
Which is why people default to sequential even when parallel is possible. They're not avoiding the join node. They're avoiding writing the join-node spec.
The unwritten spec is the real cost. The join node isn't inherently expensive. A concatenation join is basically free. A voting join with three agents doing the same task is cheap — majority wins, done. The join gets expensive when the outputs are heterogeneous and the reconciliation requires judgment.
And judgment is the thing we're least good at encoding in prompts.
Judgment requires context, nuance, tradeoffs. Prompts are bad at nuance. So the join node either becomes a full agent with a massive system prompt full of edge-case rules — which is brittle — or it becomes a human in the loop — which kills the latency advantage of parallelism.
So we've got a kind of impossibility result. Parallelism is only worth it when the join is cheap. The join is only cheap when the outputs are either homogeneous enough to vote on or heterogeneous enough to simply concatenate. The interesting cases — where the outputs are heterogeneous and require judgment — are exactly where parallelism is most expensive.
That's the paradox. The cases where you most want parallelism — complex tasks where multiple perspectives would help — are the cases where the join is hardest. The cases where the join is easiest — simple tasks where the outputs are predictable — are the cases where parallelism adds the least value.
So the sweet spot is narrow.
It's narrow but real. Research tasks with independent sources. Fact-checking where each agent checks a different claim. Translation into multiple languages. Anything where the outputs don't need to be reconciled because they're answers to different questions.
Hilbert, you've been quiet. You've got something on this, don't you?
Hilbert: Late nineties. I ran a data-cleaning shop. Insurance claims. Four temp workers, each got a batch of claims on Monday morning. They'd reconcile them against policy records, mark discrepancies, and hand their spreadsheets to a senior clerk on Friday.
A human fan-out with a human join node.
Hilbert: The senior clerk was the most expensive person in the building. She'd spend all Friday merging four spreadsheets by hand. Looking for duplicates across batches, resolving contradictory entries, deciding which temp worker had the right version when two of them touched the same claim.
And the operation ran sequential for years?
Hilbert: Sequential for years. One clerk, one batch, hand it off. Parallel was faster on paper but the merge ate all the savings. Management kept saying we should parallelize, and every time they tried, Friday turned into a disaster.
What changed?
Hilbert: We wrote the spec. Sat down and defined exactly what counted as a match across batches. Claim number, date of service, patient name. If two temps touched the same claim, the resolution policy was most recent service date wins unless the policy number didn't match, in which case flag for manual review.
You defined the reconciliation rules before the work started.
Hilbert: Once the rules existed, the merge became mechanical. The senior clerk didn't need to judge anymore. She just applied the rules. Took her two hours instead of eight.
So the join node wasn't inherently expensive. The unwritten join spec was.
Hilbert: That's what I'm saying. People default to sequential because they haven't done the work of defining what reconciliation means for their domain. They think the join is hard because they're imagining a human making judgment calls. But once the spec exists, the join is just pattern matching.
What happened to the senior clerk?
Hilbert: She got a rubber stamp. Said APPROVED — H.F. on it. Her initials. She'd stamp batches without reading them when the automated checks came back clean. Saved her for the edge cases.
That's a join node that pattern-matches instead of adjudicating.
Hilbert: The stamp only worked because the spec was so tight. If the rules said no conflict, there was no conflict. She didn't need to verify.
And the stamp was the signal that the merge was mechanical.
Hilbert: She was proud of that stamp. Had it made at a shop on Main Street. Cost her twelve dollars.
The join node as rubber stamp. That's the ideal. The join is so well-specified that it doesn't need to think.
Which is the opposite of how most people build these systems. They throw four agents at a problem and then throw a fifth agent at the merge with a vague prompt about reconciling differences.
Hilbert: That's what we did the first three times. The fifth agent — well, the senior clerk — burned a whole day and the output wasn't even consistent week to week.
The spec made it consistent.
Hilbert: The spec made it stampable.
Hilbert, you said the stamp was for clean batches. What happened when the batch wasn't clean?
Hilbert: Then she earned her salary. The stamp stayed in the drawer and she went line by line. But clean batches were about eighty percent of the volume. The stamp handled the common case. Her judgment handled the exceptions.
That's the architecture. Cheap join for the common case, expensive join for the edge cases. You don't need the expensive join on every run.
The spec tells you which is which before you pay for the expensive join.
Hilbert: That's the part people skip. They don't write the spec. They just pay for the expensive join every time and wonder why parallel costs more than sequential.
The spec is the thing that tells the join node when it can stamp and when it has to think.
Hilbert: We had a name for batches that looked clean but weren't. We called them—
Don't tell me. Stamped anyway?
Hilbert: Stamped anyway. Took us six months to find the first one. Claim for a procedure that didn't exist yet. The temp had fat-fingered the date and the spec said most recent service date wins, so the fake date won.
The spec was correct but the input was wrong. No spec catches bad inputs.
Hilbert: The spec catches contradictions. It doesn't catch fabrications. Different problem.
That's the thing about the join node. It can only reconcile what it receives. If all four agents are confidently wrong in compatible ways, the join node stamps it.
Garbage in, stamped garbage out.
The misconception here is that parallel is always better when the dependency graph permits it. The join-node reconciliation cost can exceed the latency savings, and the operational headaches — debugging, error recovery, the unwritten spec — push you toward sequential as the pragmatic default.
The second misconception: a loop in a multi-agent system is a cycle in the graph topology. Most real systems implement loops as a controller outside the graph that re-runs it until a condition holds. The graph is a pure function. The loop is in the orchestration layer.
The third one — the join node is just a concatenation step. When agents return contradictory results, the join node is doing adjudication, and that reconciliation logic is a spec you have to write. Often harder than writing the agents themselves.
The open question is whether that flips. As join-node reconciliation becomes more standardized — voting schemes, confidence thresholds, source-authority hierarchies — does the cost curve shift and make parallel the default?
The controller pattern is winning right now because it's testable. But the next frontier is probably in-graph cycles with well-defined exit conditions, once the tooling catches up.
The reframe to leave with: the loop was never the object. The topology is. And the topology is a decision about where you put the intelligence — in the graph or outside it.
Thanks to Hilbert Flumingtop for producing.
This has been My Weird Prompts. If you want more on graph topology and agent architecture, the website is my weird prompts dot com.
We'll be back soon.
