I want Claude Code doing the difficult thinking through Anthropic, but handing suitable work to a model on my local network. Can a proxy actually recognize those delegated requests? Or am I better off giving Claude a tool that explicitly calls Ollama?
The appealing part is keeping Claude in charge without paying Claude to type every docstring. The less appealing part is discovering that your local GPU has been ornamental.
There are working implementations of both paths. The proxy path can split the traffic, but it doesn't recognize a subagent by some special identity badge. It reads the requested model name. You arrange for subagents to request a local model, and the proxy sends that name downstream.
Routing by name tag. Enterprise architecture returns to kindergarten.
That distinction matters to me. I don't want something guessing that a short request must be a subagent. I want the main conversation staying on Anthropic.
Then model-based routing is the concrete mechanism to examine. Explicit delegation through MCP is the other clean option. Neither requires a classifier trying to infer which request looks sufficiently junior.
Herman, before you fall in love with the two-function router, does Claude Code officially support choosing a separate local endpoint for a subagent?
No. The feature request for that was opened August twenty-third and closed for inactivity October fifteenth. It asked for local OpenAI-compatible endpoints as subagent targets. The implementations we're discussing are third-party integrations, not a native per-agent provider setting.
Closed for inactivity. The local GPU has company.
The subtlety is that Claude Code can still emit a custom model string through these integrations. That's enough for a proxy to exploit, even though there isn't a supported provider object where you simply assign this agent to that server.
Meaning the distinction exists because I create it in configuration, rather than because Claude Code sends a reliable subagent header.
Yes. I haven't seen a demonstrated header or request-path discriminator in these implementations. The demonstrated distinction is the model field. If main and worker requests carry the same model name, this particular router cannot tell them apart.
I prefer that answer to a router reading the prompt and deciding whether it sounds important. My least important requests are usually my longest.
You would defend a router that waits patiently and does nothing.
Only if its metrics admit it.
They do, actually. The small Go project called llm-router exposes separate counters for Anthropic requests, downstream requests, and errors. You can check the split instead of deciding that a warm laptop means local inference is happening.
Walk me through that setup as something I could actually build. Where does the interception happen?
Claude Code's Anthropic base URL points to the proxy running on your machine. The proxy's downstream address points to Ollama, either on that machine or on the LAN box. Claude model names pass through to Anthropic. Other model names go downstream.
Its routing logic is unusually legible. One function extracts the model from the JSON body. Another checks whether that name starts with the Claude prefix or the Anthropic provider prefix, ignoring case. That's the decision. It doesn't grade the prompt or inspect the difficulty of the code.
You sound relieved that software has only one opinion.
I am relieved that I can locate the opinion.
Where do I tell Claude Code to use the other name for workers?
Through its subagent-model environment setting. In the documented arrangement, the main session remains on Opus, while the subagent default names your Qwen coder model. Claude Code sends that string to the proxy, which recognizes it as downstream traffic.
The dangerous word there is default.
Very dangerous. A subagent definition must omit its own model field to inherit that custom subagent default. Put an explicit Sonnet selection in its frontmatter and that agent stays on Anthropic. You can build a mixed fleet deliberately, but an old model declaration can also silently defeat your local routing.
That's exactly the kind of thing I was wondering about. The proxy could be working perfectly while the configuration never sends it the model name I expect.
A successful request can be an unsuccessful experiment.
Your observable check is the downstream counter moving when the intended worker runs, plus activity on the local server. A green health endpoint only establishes that the proxy responds. Corn, would you trust a task completing as evidence that it ran locally?
No. Completion proves someone did the work. I'd want the chosen backend recorded beside the request. Particularly if a cloud fallback can make a broken local setup look healthy.
That matters again later with privacy. Silent recovery can erase the very property you wanted from the system.
Are there other settings involved beyond those addresses and the worker model?
The published example also enables experimental agent teams and configures a custom model option for the model picker. Those are parts of that recipe, not proof that every delegation workflow needs agent teams. I would reproduce the documented configuration first, then remove what your actual workflow doesn't require.
You normally remove things after the terminal stops opening.
That is why I now appreciate reproducible configurations.
The proxy runs as a single Go binary with no external dependencies. It passes Claude Code's authentication headers through rather than storing credentials. That still puts it directly in the credential path. Not storing a key does not mean never seeing a key.
Does that make a tiny router preferable to a bigger routing platform?
Easier to inspect, certainly. Not automatically trustworthy. Two functions can send your secrets to the wrong place just as efficiently as two thousand.
This one has body limits, timeouts, rate limiting, and hop-by-hop header filtering. Useful engineering, but they answer different questions from whether its upstream selection and authentication handling are correct. Its stated motivation includes the LiteLLM package compromise in March, when a credential stealer reached the Python package supply chain.
An API-key proxy is a particularly ambitious place to install surprises.
A more elaborate pattern is subagent-fleet. You describe a topology of roles and machines, and it generates routing configuration and agent definitions. Its example puts planning on a laptop, implementation and review on a Mac mini, and summarization on a workstation. It adds health monitoring, model warmup, and streaming traces.
That sounds closer to a little inference cluster than one Ollama machine.
It is. Useful if you already have several suitable machines. Unnecessary moving parts if you're proving whether one worker can reliably generate tests.
Start with one idle GPU. Expand to several once confidence improves.
The tool-based approach sounds less dependent on Claude Code's model-selection behavior. What gets installed there?
Local-vibes is a concrete example with three distinct pieces. An MCP server provides a delegate tool. A skill tells Claude when delegating is appropriate. A prompt-submission hook nudges it when your request includes language such as offloading or running something locally. None of that edits your main Claude instructions file.
Three removable pieces. Herman's preferred definition of temporary.
Removable matters, Corn. If I disable the MCP server and its accompanying guidance, the primary session doesn't require a different API transport to remain useful.
Does that mean I could use the same approach with a Max subscription, without replacing the Anthropic base URL?
Yes. The main session stays where it is. Claude makes an ordinary MCP tool call, and the server calls the local model. It works as delegation through a tool rather than transparent replacement of a native Claude subagent.
That difference deserves keeping. A tool wearing an agent's hat doesn't inherit every permission and behavior of a Claude Code subagent.
The delegate call supplies a task and working directory, plus controls such as read-only mode and execution lane. Inside the server, the local model runs its own agent loop. It can list directories, search, read, edit, write, and execute shell commands.
Then it really can do the job, rather than just returning a proposed answer for Claude to paste into files.
In that design, yes. It returns a summary and a report of changed files. The loop has a default iteration limit of twenty-five, and shell calls have a two-minute timeout. Those bounds stop one delegated task wandering indefinitely.
Two minutes is plenty of time to make a regrettable directory.
Its file operations reject path escapes outside the working directory. However, the project itself describes shell access as gated, not jailed. A working-directory check on file tools does not turn arbitrary shell execution into a secure operating-system sandbox.
I might prefer something where the local model cannot modify anything initially.
That's my starting preference too. Let it propose a change while the expensive model retains the pen. Although... no, that doesn't automatically make the result cheap. Claude still has to read it.
Local-llm-mcp takes precisely the text-only route. Its local-model tool returns text and gives the model no repository tools. The author calls that a deliberate boundary, not an unfinished feature. Corn, would that restriction make you use it, or merely make you less opposed?
For regex candidates, small transformations, or alternative implementations, I'd use it. For summarizing files it hasn't actually received, I'd be less impressed. Restricting tools changes what evidence the worker can gather, not just what damage it can do.
That's an important limitation. You must supply the relevant material or put file-reading logic in the server. A text-only model cannot independently investigate missing context.
Is there a middle option, where the tool knows what task it's doing and reads the files itself?
Ollama-code-mcp offers task-specific tools for code generation, review, refactoring, fixes, tests, explanations, and batch changes. Its file-aware variants read files server-side. Claude can identify the files instead of copying their entire contents into the tool arguments.
Finally, delegation without first dictating the filing cabinet.
That can preserve primary-session context. It also requires that the server can actually access the repository. An Ollama server on your LAN does not magically acquire the files from your laptop.
Right. I need to distinguish the machine serving inference from the process running the MCP tools.
The tool server can run beside your checkout and call Ollama remotely. If you instead run the tool server remotely, it needs a suitable checkout or mounted directory. Otherwise your beautifully specified working directory refers to a folder on the wrong computer.
Distributed systems remain committed to misplaced folders.
The task-specific project also returns a non-fatal failure when Ollama is unreachable, so Claude can do the work itself. That's convenient operationally. Whether fallback is acceptable depends on why you selected local processing.
For token savings I might accept fallback. For something explicitly kept local, I wouldn't.
Then failure should remain visible. I don't want a privacy setting that becomes a suggestion when the Wi-Fi sulks.
Delegate-local explicitly pins local-designated models to local failover chains to prevent a silent jump to cloud inference. It also tries to reuse stable agent prefixes across batched tasks, allowing local prompt-cache reuse. The project's claimed prompt savings there are local inference savings, not a measured reduction in your Anthropic bill.
That distinction is easy to lose in a headline. What hardware am I actually allocating to this delegated layer?
The main Claude session needs no model-sized GPU because the inference is remote. Your local layer must hold model weights, runtime overhead, and the attention cache for active contexts. More concurrent agents consume more memory even when they share one loaded model.
Eight workers do not fit merely because eight sounds productive.
Local-vibes allows bounded concurrency, but a software semaphore isn't a promise your GPU can sustain its default. Start with a single task and observe memory, throughput, and queueing. Especially if the local model shares system memory with your editor and everything else.
How small can I go before this becomes an exercise in watching a spinner?
One reported evaluation ran Qwen coder models of seven and fourteen billion parameters on a laptop GPU with twelve gigabytes of memory. That's evidence that useful bounded tasks can run on consumer hardware. It doesn't establish a comfortable context length or fast concurrent repository editing for every configuration.
Twelve gigabytes buys a worker, not a department.
For more substantial coding models, twenty-four gigabytes recurs in practical examples. One rented setup used a twenty-seven-billion-parameter model in a quantized file of about seventeen gigabytes. The gap between the file and the card's capacity is needed for context and execution, not decorative headroom.
Would a Mac with twenty-four gigabytes of unified memory be equivalent?
Not in available model memory. The operating system and applications share that pool. The author of the text-only MCP project froze a twenty-four-gigabyte Mac with local models and added a free-memory valve, plus an exclusive-backend guard so heavy servers don't compete.
The intern cannot touch the repository, but can evict the desktop.
The valve's nominal default reserves sixteen gigabytes free, although its setup adjusts the threshold on smaller machines. You have to inspect the actual selected setting. A sensible guard for a large machine can refuse nearly everything on a small one.
I don't necessarily need to buy a machine before trying this, then. I could rent an inference box just to test the workflow.
One orchestration experiment did exactly that with a rented twenty-four-gigabyte workstation GPU. The reported GPU spend was about forty cents for the session. Useful for testing model suitability before buying hardware, though renting changes the privacy assumptions of the word local.
Local to somebody's electricity meter.
A larger unified-memory machine gives you room for larger weights or contexts. Subagent-fleet's example uses a sixty-four-gigabyte Mac mini for implementation and review. That is an example topology, not a minimum requirement.
Which jobs actually justify all this? I can imagine boilerplate, but I don't want a worker producing code that takes longer to verify than to write.
I'd choose by how cheaply the result can be rejected. A format conversion with a schema is promising. A confident explanation of an unfamiliar security module is not, even if the explanation is only one paragraph.
The recurring practical list includes scaffolding, docstrings, README drafts, commit messages, regex candidates, test skeletons, and repetitive refactors. Long-file summaries and first-pass diff review also appear frequently. Those latter tasks save reading, but their omissions can be difficult to detect.
What about asking it to search a whole repository and tell Claude which files matter?
Use literal search first when the condition is literal. Asking a model to find an exact identifier is a costly way to introduce an opinion about spelling.
A local model becomes useful when the search criterion is semantic, such as identifying where retries may duplicate writes. Even then, ask it for locations and supporting snippets, not just a reassuring conclusion. Claude can inspect a small candidate set rather than ingesting the entire repository.
That's probably more interesting to me than getting a cheaper commit message. The context reduction could matter during a long session.
It may matter more than the bill. File-aware delegation can keep bulk inputs and intermediate exploration out of the primary conversation. Returning a compact, evidence-bearing result preserves room for the decisions Claude actually needs to make.
Provided compact doesn't mean leaving out the one awkward file.
Would you delegate test generation, Corn? You often prefer something runnable over a polished explanation.
Skeletons, yes. The authority to decide whether the implementation is correct, no. Generated tests can faithfully reproduce the same misunderstanding as generated code. I'd rather give the worker existing independent checks than ask it to invent its own acquittal.
The best measurement here used deterministic gates. In the Claude Code feature request, an author reported sixty-six bounded tasks accepted or rejected by executable checks. The fourteen-billion-parameter model achieved seventy-seven percent acceptance overall.
Acceptance means the gate passed, rather than Claude liking the answer?
Correct. The smaller model accepted roughly three-fifths of the tasks. The larger model improved coverage, but not uniformly. Its SQL tasks all passed, while test generation passed only half.
A very confident employee in two adjacent departments.
These are small task families, not universal model capabilities. Still, they support selecting work by the strength of the checker. A seemingly difficult code transformation can be safer to delegate than a simple classification whose errors nobody can detect.
Does seventy-seven percent accepted translate into anything like seventy-seven percent fewer Claude tokens?
No. It measures coverage of that task set. You still have task formulation, tool calls, failed attempts, fallbacks, and whatever verification the main session performs. Accepted work also varies enormously in size.
Seventy-seven percent of errands is not seventy-seven percent of rent.
Then how should I think about the actual savings?
Compare the cloud work avoided with the cloud work added. You avoid Claude generating and perhaps exploring the delegated task. You add a delegation prompt, result ingestion, review, and sometimes repair. If Claude receives a large answer and re-derives it to be sure, you've moved the typing but retained much of the thinking.
You hired a junior and billed the senior for watching.
The read-back tax can consume most of the apparent saving. Output tokens avoided and input tokens added also have different prices and caching behavior. Counting raw tokens alone can mislead you even before you count wall-clock time.
I suppose having Claude review everything is the obvious cautious setup, but that could defeat the financial point.
An orchestration comparison makes that tension visible. Claude broke the work into small tasks, reviewed results, and sent defects back with reproductions. The orchestrated runs finished at seventy-eight and seventy-seven checks passed, out of seventy-eight.
Almost clean, after supervision.
The one-shot runs scored lower. More revealingly, the same replay bug appeared in both approaches initially. Replaying a log wrote the entries again. Review caught it in the orchestrated runs. Splitting the task did not, by itself, prevent that defect.
Then the evidence supports the reviewed workflow, but doesn't isolate the benefit of delegation.
That's the author's own limitation. The comparison combines task decomposition, review, and repair. It cannot tell us how much decomposition alone buys. The tiny GPU bill also excludes any inference spent by Claude orchestrating those turns.
Forty cents for the worker. Supervisor sold separately.
Subagent-fleet reports another encouraging but limited comparison. Across eight coding tasks, its fleet averaged about eight point four against Sonnet's eight point nine, judged blind by a model. It describes that as roughly ninety-four percent of the quality.
That sounds promising, although a quality score isn't necessarily a useful ratio.
Nor does eight tasks establish a personality.
The fleet also took roughly seventeen seconds versus five for Sonnet in that comparison. Its listed local inference charge was zero, which means no metered provider charge in that table. Hardware, electricity, and maintenance did not disappear.
Could the deterministic gate remove most of that review cost?
For a narrowly specified result with strong independent checks, yes. Not because a compiler proves every requirement, but because the gate can settle the particular contract you've delegated. A parser accepting JSON proves syntax. A schema adds structure. Neither establishes that every extracted fact is correct.
The gate is only as demanding as the person who built it.
A transformation with comprehensive fixtures might return a pass status and changed-file summary. Claude needn't re-read every unchanged byte. A security-sensitive change with a weak test suite deserves much more inspection, regardless of which model wrote it.
That's a more useful distinction than cheap model versus expensive model. I need to know where I can stop paying for repeated verification.
Corn, would you let a worker's successful health check stand in for checking where the task actually ran?
...only if it accounts for the Portuguese. My aunt Vera's email replies are going through my inference box this month. The coding worker stays on Anthropic because I put an explicit model in its agent definition. The GPU is idle except when Vera answers the residents' association.
Separate routing rules for correspondence. An ambitious starting point.
It's meant to draft replies, not send them. I type the Qwen coder name with a Portuguese suffix because that's the alias Vera gives me. The alias points to her Portuguese instruction preset. For three weeks I think somebody is interfering with the mail. Now she chairs the association's Portuguese correspondence committee. None of the residents speak Portuguese.
The extra suffix is your local alias, then. The router has no knowledge of what that alias loads.
Yes. Your two-function explanation is very tidy, Herman. Vera's configuration is not. My cousin Martin is looking at it every Thursday. He unplugs the box, reconnects it, and charges four hundred dollars. I pay cash. The receipt is on a napkin because he says this isn't a business visit. The keyboard isn't made for an anteater's claws, so Vera types the address while I hold the cable. I keep her draft window enlarged. She doesn't like hunting for the reply button.
Four hundred dollars for recovery, with the original routing intact.
The drafts are useful. The language is the difficulty. Martin says changing that is another visit. Meanwhile the association is asking Vera for an English translation of its own parking notice.
You could separate the draft generator from the mail workflow entirely. Then a routing mistake would produce an unwanted draft rather than altering the correspondence process.
It already only drafts. Vera copies them herself. That's why Martin says the connection is working.
A draft that someone manually promotes can still travel a long way. I'd want the actual selected backend and instruction preset in the result, not buried in a startup log.
That applies to my setup too. A task summary saying done wouldn't tell me whether I saved cloud work or silently fell back.
Exactly the operational distinction to measure. Well... the distinction, not a reason to add another model to audit the first model. Record backend selection, cloud input and output usage, local duration, acceptance, and fallback. The text-only MCP project ships a benchmark harness instead of promising everyone the same saving.
I'd also count how often I abandon the worker and do it directly. Failed experiments have a habit of disappearing from the success table.
Would you recommend the tool approach first for me, rather than the proxy?
For proving useful delegation, yes. Start with a text-only or read-only MCP tool and a bounded task with independent checks. Your main Claude session stays untouched, and you can inspect exactly what was handed over. Use proxy routing when preserving the native subagent workflow is itself a requirement worth the configuration fragility.
I'd keep one deliberately cloud-backed agent too. Not as hidden fallback. As an explicit choice for tasks where correction costs more than generation. That makes the boundary something you can reason about.
The proxy is possible, then, but it isn't discovering subagents. I'm assigning a model name that serves as a routing label.
That's the mechanism. One adjacent detail worth keeping separate: Ollama added subagent and web-search support in its Anthropic-compatible layer in February. That helps running Claude Code against Ollama, but doesn't itself establish the selective cloud-main, local-worker split we're discussing.
Compatible transport doesn't choose who does the work.
Nor guarantee that a chosen model reliably performs the required tool calls. A coder-tuned model with the appropriate template matters as much as getting requests to the right port.
I'd be interested in native support making those choices explicit, especially if the result could say which backend actually handled it.
That's what I'd watch for: delegation that returns both a checkable result and an honest account of where it ran. The context savings could then be valuable without making every completed task a small forensic exercise.
Thanks to Hilbert Flumingtop for producing the show.
If you want more of this, try episode two, Local STT For AMD GPU Owners; episode seventeen, Cloud Render Superpowers; and episode twenty, Architectural AI. This has been My Weird Prompts, the human-AI collaboration podcast.
If this helped untangle your own setup, leave us a review wherever you listen.
Thanks, both. See you soon.
See you tomorrow.
