localagents
Hand Claude Code's grunt work to a model running on your own hardware.
localagents is an MCP server that gives Claude Code a run_agent tool. Each call
starts a full headless Claude Code session — same tools, same CLAUDE.md, same
working tree — except its API traffic goes to a llama.cpp or vLLM server you run
instead of to Anthropic. Claude writes the brief, the local model does the work,
Claude reviews the result. Your Anthropic token budget goes on the parts that
need it.
A 27B Qwen on one GPU is perfectly capable of "add a CLI for this module and tests to match"; Opus is better spent on the design conversation than on watching pytest run. Two local agents in parallel can build the two halves of a package against a pinned interface.
Status: early. It works, I use it daily, and the interface will move. It targets llama.cpp and vLLM specifically; Ollama isn't a goal.
How it works
Claude Code (your session)
│ MCP: run_agent(task, model=...)
▼
localagents ── spawns ──▶ headless `claude` (Agent SDK)
│ │ ANTHROPIC_BASE_URL
│ ▼
└──── in-process shim ◀────┘ normalises requests, logs them,
│ translates backend errors
▼
llama-server / vllm (/v1/messages, on your machine or your LAN)
Three things make this more than an environment variable:
-
Endpoints, not models, in the config.
models.yamlis a list of servers. What each one is actually serving right now, its real context window, and how many slots are busy are discovered on every call via/v1/modelsand friends. You bring models up and down by hand — the server never launches anything — and Claude uses whatever is up. If nothing suitable is running it asks you to start something. -
A shim between Claude Code and the backend. Claude Code sends things local chat templates reject, and local servers fail in ways Claude Code doesn't recognise. The shim fixes both directions (details below) and writes a
requests.jsonlper job so you can see exactly what went over the wire. -
The same isolation model as Claude's own subagents. By default a job works in your tree, like the Agent tool does.
isolation: worktreegives it a fresh git worktree on alocal-agent/<job>branch, kept only if it changed something, with a diffstat in the job record so Claude can review it as a diff.
Requirements
- Python 3.12+ and uv
- Claude Code. The Agent SDK bundles its own
claudebinary, so nothing else to install. - A server that speaks Anthropic's
/v1/messages:- llama.cpp
llama-server— start it with--jinja; add--slots --metricsto get occupancy and cache stats in the tool output. - vLLM with
--enable-auto-tool-choice --tool-call-parser <parser>.
- llama.cpp
- A model that can actually drive Claude Code: solid native tool calling and a context window of 128k or more per request. Qwen3.8-27B works well. Smaller windows work but compact constantly; see Context windows.
Install
git clone https://github.com/ccebelenski/localagents.git && cd localagents
uv tool install -e . # `localagents` on PATH; editable, so repo edits apply
cp models.example.yaml models.yaml # edit for your servers (gitignored)
claude mcp add --scope user local -- localagents --config "$PWD/models.yaml"
User scope means every project gets the local server. It inherits the cwd of the
Claude Code session that launched it, so run_agent defaults to that project's
tree. A project can carry its own ./models.yaml to override the registry.
If you'd rather keep it to one project, put this in that project's .mcp.json:
{"mcpServers": {"local": {"command": "localagents", "args": ["--config", "/path/to/models.yaml"]}}}
Restart Claude Code (or /mcp → reconnect) after adding it; MCP servers load at startup.
Then install the local-agent subagent, so jobs show up as agents (see below):
mkdir -p ~/.claude/agents && ln -s "$PWD/agents/local-agent.md" ~/.claude/agents/
It names the server's tools as mcp__local__*; if you registered the server under
another name than local, copy the file instead and change the tools: line.
Using it
Claude picks it up like any tool. Ask for it by name and it'll do the right thing:
Use the local agent to add a
--jsonflag to the CLI and cover it in the tests.
What Claude does behind that: list_models to see what's up and pick a model, then
run_agent with the brief and wait_s set to the job's timeout. The call blocks
until the job ends; after two minutes Claude Code moves it to a background task (shown
in /tasks), Claude keeps working, and the final job record arrives as a task
notification. Claude reads files_touched (or the worktree diff) and checks the work.
The server sends a progress notification every minute while it waits, so the call never
trips Claude Code's idle timeout, and the tool timeout itself defaults to about 28 hours.
The two minutes are Claude Code's default (CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS, in
milliseconds). MCP calls run one after another, so with several agents to launch that
is two minutes each. Two ways round it, which combine:
- Set the variable lower in the
envblock of your settings.10000backgroundsrun_agentafter ten seconds and leaveslist_modelsand the other quick calls inline; alocal_completethat runs long comes back as a notification instead. - For a swarm, start each job with
wait_s: 0(returns the id at once) and make onewait_jobscall for all of them: one wait, one notification, however many jobs.mode: "any"returns on the first to finish, so Claude can review it and call again on the rest.
If nothing suitable is running you'll be asked to start one:
No live endpoint is serving a model matching
qwen. Ask the user to start one. Currently serving: {'dgx1': ['GLM-5.3-Flash-EXL3']}
Start whatever you like, however you normally do, say "it's up", and Claude retries.
Which model that is, is your call; the config never names one. The flip side: anything
you leave running is fair game. Claude is told to name the model it's about to use and
to check with you if the id looks unfamiliar, and list_models flags windows under
128k, but the real gate is what you choose to start.
Tools
The local-agent subagent
Auto-backgrounding needs Claude Code 2.1.212 or later and is off inside subagents and
in non-interactive runs unless CLAUDE_AUTO_BACKGROUND_TASKS=1. Without it a
run_agent job is an MCP tool call plus a job id Claude has to remember to poll:
Claude Code doesn't know it's an agent, nothing tells Claude when it finishes, and a
job can be forgotten if Claude ends its turn before it's done.
agents/local-agent.md covers those cases by making each job
a real background subagent. It's only a relay. It runs on Haiku, can use only the job
tools (run_agent, wait_job, job_status, job_log, cancel_job), passes the
brief through verbatim, waits, and returns the final job record unchanged. Picking the
model stays with the main session, which still has all of the server's tools,
list_models included. Claude passes its choice to the relay in a short header:
model: unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL
isolation: worktree
---
Add a --json flag to the CLI and cover it in the tests. ...
model: is required and should be the exact served id from list_models. Which
model runs matters, and the main session is the one with the live context windows
and slot occupancy, the conversation, and your instructions. The relay doesn't have
any of that, so it won't fill in a model, fall back to a default, or swap one. With
no model: it refuses to start, and if the model isn't up it returns the
model_unavailable error so Claude asks you as usual. Other header keys: endpoint,
cwd, max_turns, timeout_s, permission_mode, resume_job. A resume header is
the one case where model: can be left out: a resumed job continues on the model that
ran the original session.
While it runs, Claude can still check on it directly with list_jobs / job_status / job_log.
Messages sent to the relay let Claude cancel the job or, after it finishes, continue
the same local session (resume_job). A follow-up runs on the same model, in the same
directory and isolation: a worktree job continues in its tree, and if the tree was
removed because the job changed nothing, it is recreated at the same path from the
same base commit. Jobs live in the server's memory, so a follow-up is only possible
while that server process is up. Stopping the relay from /tasks does not
stop the local job; use cancel_job. The overhead is a few small Haiku turns per job,
one per 25 minutes of job time (each wait_job blocks for 1500 s with progress
notifications in between; v0.1.5 waited 100 s at a time, which with the relay's turn
cap ended jobs longer than about 2.7 hours).
Status line
Jobs started this way are MCP tasks, not subagents, so they don't appear in the agent
panel. /tasks shows the latest progress line the server reported (status, elapsed,
turns, tool calls, last activity). For an always-visible view, the server publishes
each job's state to ~/.local/state/localagents/status/<pid>.json on every change,
and localagents statusline prints them as one line:
local: ⟳ Qwen3.8-Flash-Next 1m18s 4 calls ctx 31% 82 t/s · llamacpp 1/1 slots · +1 other
Per running job: elapsed time, tool calls, files touched, how full the local model's
context window is (ctx, from the token usage the shim sees on every backend
response, so it tells you how close the agent is to compaction) and its recent
generation speed. Then the load of each endpoint your jobs run on, probed every ten
seconds while jobs run: slots busy for llama.cpp, requests running and queued and KV
cache fill for vLLM. Finished jobs stay on the line for two minutes with a check mark
(or a cross, clock or null sign for failed, timed out and cancelled).
The line is scoped to the Claude Code session asking: each server records the Claude
Code process that spawned it, and the renderer finds its own by walking up from the
status line script. Jobs of other sessions on the same machine fold into +N other,
because the boxes are shared and knowing one is busy matters even when the job isn't
yours. Files of servers that have exited are ignored and removed.
The renderer imports nothing heavy and runs in well under 100 ms. Append it to your
status line script and set refreshInterval in the statusLine settings block so
the elapsed times tick while the session is idle:
local_jobs=$(timeout 2 localagents statusline 2>/dev/null)
[ -n "$local_jobs" ] && line+=" $local_jobs"
Job records
Job records live in ~/.local/state/localagents/jobs/<job>/: transcript.txt
(what the agent said and did), events.jsonl (every SDK message), requests.jsonl
(every backend request with timing, size and usage), and requests_full.jsonl if
you turn on request dumping.
Config: models.yaml
Start from models.example.yaml. It's re-read on every call, so edits take effect
immediately, and the server never rewrites it — register_endpoint writes to a
sidecar models.local.yaml that is merged on top.
endpoints:
llamacpp:
base_url: http://127.0.0.1:8080
backend: llama.cpp
gpu-server:
base_url: http://gpu-server.lan:8000
backend: vllm
host: gpu-server
- endpoints are places that serve
/v1/messages. That's the whole config: what each one is serving is probed, never written down. Order is priority — with no model named, the first endpoint that is up and serving something wins.hostandnotesare informational and shown inlist_models;envadds environment variables to sessions that run against that endpoint. - Picking a model is done per call.
run_agent(model=...)takes a served id, a glob, or a fuzzy name (qwen3.8-27bfindsunsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL);endpoint=pins a server. Both are only needed when more than one thing is up.defaults.modelis an optional standing preference in the same syntax. - defaults also cover
permission_mode(acceptEdits), allowed and disallowed tools (subagents can't spawn subagents), which Claude settings to load,max_turns,timeout_s, and a system-prompt suffix telling the agent it's a delegate and how to report back.
A models: section from an older config is ignored.
What the shim does
Both llama.cpp and vLLM speak /v1/messages natively, so pointing
ANTHROPIC_BASE_URL at them almost works. The shim closes the gaps:
System messages mid-conversation. Claude Code puts role: system entries
inside messages — the skills listing, a token-budget marker, and one more per
turn. Qwen's chat template refuses: "System message must be at the beginning".
The shim folds each one into the adjacent user message as a <system>…</system>
text block, in place. Hoisting them into the top-level system field instead
changes the start of the prompt every turn, which invalidates the server's
KV-cache prefix and re-evaluates the whole ~35k-token prompt each time (21–47 s
per turn on a 27B). Folding in place keeps the prompt append-only: f_sim_best
0.88–0.99 in llama-server's log, 2.5–14 s per turn.
Text next to tool results. Claude Code appends a <system-reminder> text
block to the user turn that carries tool results (and the fold above adds a
<system> block there too). Gemma 4's chat template renders a tool response
only when it sits directly after the assistant's tool call; behind llama.cpp's
/v1/messages converter that mixed turn breaks the adjacency and the model
sees none of its tool output — it then invents file contents, every Edit
misses, and it reports success anyway. The shim moves such text blocks inside
the last tool_result's content, which every template renders. Measured on
gemma-4-31B-it-qat (llama.cpp b11515): prompt growth after three file reads went
from 256 tokens (blind) to ~3.1k (sees them); Qwen3.6 and Muse-Glimmer were
unaffected either way.
Context overflow. Claude Code assumes a 200k window for any model it doesn't
recognise; with a smaller slot it runs into llama.cpp's
exceed_context_size_error, which it doesn't understand, and the job dies. See
the next section.
Everything the shim does is a no-op when it isn't needed, and every request is logged with its timing, message count, byte size and reported usage.
Context windows
Two layers keep a session inside the real window:
- The probe reads it — llama.cpp
/propsn_ctx(per slot:-cdivided by--parallelwhen unified KV is off), vLLMmax_model_len— and the session getsCLAUDE_CODE_MAX_CONTEXT_TOKENS. Claude Code's own auto-compact then fires at the right point. Below 128k the output budget is also shrunk ton_ctx/8, because the compact threshold iswindow − max_outputand would otherwise sit at zero. - If a request still overflows, the shim rewrites the backend's error into
Anthropic's
prompt is too long: N tokens > M maximum, which Claude Code answers by compacting and retrying.
At 64k this works but thrashes: Claude Code's ~20k of fixed prompt and tool schemas, plus a ~7k-token compaction summary (~55 s on a 27B) and the files it re-attaches, refill the window within a few turns and its thrash guard ends the job. Give each slot 128k or more.
Backend notes
- llama.cpp:
llama-server -hf <gguf> --jinja -fa on --slots --metrics, plus--reasoning onfor thinking models and--parallel Nfor concurrent jobs. With/slotson,list_modelsshows{total, busy, free}so Claude knows whether a second agent will run now or queue. With/metricson, each job records prompt tokens processed vs. cached, cache hit ratio, prompt and generation tok/s, and speculative-decode acceptance — the counters are server-wide, so overlapping jobs share the delta. - vLLM:
vllm serve <model> --served-model-name <alias> --enable-auto-tool-choice --tool-call-parser <parser>. The context window comes frommax_model_lenin/v1/models; occupancy (requests_running/waiting,kv_cache_usage) and the per-job cache stats come from its always-on/metrics. vLLM reports context overflow on the Anthropic endpoint as HTTP 500internal_error; the shim translates that too. The session is started withCLAUDE_CODE_ATTRIBUTION_HEADER=0because the per-request attribution hash defeats prefix caching. Verified with DeepSeek-V4-Flash: thinking blocks replay with their signatures, prefix cache hits on every turn after the first. - The first turn of a job costs about 20k tokens of prompt on a cold slot (system prompt plus tool schemas), ~10 s on a 27B. Everything after that is a cache hit plus the delta.
Development
uv sync --dev
uv run pytest -q
See CONTRIBUTING.md for the layout and how to test changes
against a real server.
License
MIT. See LICENSE.
Copyright © 2026 Chris Cebelenski
