Computer Use is an open-source MCP server — a computer-use agent (CUA) backend — that lets any coding agent use your computer the way a person does. It reads the screen through accessibility trees, clicks and types in the background so your mouse stays yours, shows an agent pointer where it is working, zooms in on small text, and drives tabs in your signed-in Chrome — on .
Open-source Computer Use MCP server for any coding agent — macOS, Windows, Linux. Accessibility-first, background input, agent pointer, zoom, your signed-in Chrome.
COMPUTER_USE_BROWSER, COMPUTER_USE_AGENT_CURSOR
上次检查
2026-11-09
星标
35
工具
—
可用率
—
macOS, Windows and Linux
Works with Claude Code, Codex, Cursor and MT Code, or any other MCP client, with any model — no vision model is required for interaction.
Built by Munim Technologies as the Computer Use engine of MT Code, and published here on its own.
Download the latest binary for your platform from Releases (munim-computer-use-macos-universal.zip, munim-computer-use-windows-x64.zip) or build from source.
Put it somewhere on your PATH (/usr/local/bin/munim-computer-use, or %LOCALAPPDATA%\Programs\munim-computer-use\munim-computer-use.exe).
macOS only: run munim-computer-use request-permissions once to be prompted for Accessibility and Screen Recording.
Register it with your agent:
# Claude Code — fastest: the npm launcher fetches the signed binary on first run.
# Add `--scope user` to register it for every project instead of just this one.
claude mcp add munim-computer-use -- npx -y munim-computer-use
# or point at a downloaded binary
claude mcp add munim-computer-use -- /usr/local/bin/munim-computer-use
# Codex — writes the entry below into ~/.codex/config.toml for you (needs a Codex CLI
# with `codex mcp`; check with `codex mcp --help`)
codex mcp add munim-computer-use -- npx -y munim-computer-use
# Codex — ~/.codex/config.toml, if you would rather edit it yourself
[mcp_servers.munim-computer-use]
command = "npx"
args = ["-y", "munim-computer-use"]
# Cursor — no MCP subcommand in its CLI, so write the config. ~/.cursor/mcp.json applies
# to every project; .cursor/mcp.json in a repo applies to that one.
mkdir -p ~/.cursor && [ -s ~/.cursor/mcp.json ] || echo '{}' > ~/.cursor/mcp.json
jq '.mcpServers["munim-computer-use"] = {"command":"npx","args":["-y","munim-computer-use"]}' \
~/.cursor/mcp.json > ~/.cursor/mcp.json.tmp && mv ~/.cursor/mcp.json.tmp ~/.cursor/mcp.json
// Cursor — the entry that produces, in .cursor/mcp.json
{ "mcpServers": { "munim-computer-use": { "command": "npx", "args": ["-y", "munim-computer-use"] } } }
Then ask: "Open Safari, find the cheapest flight to Denver on Tuesday and put it in a note." The agent reads the UI with get_app_state, acts by element id, and verifies with screenshot.
Capability matrix
Munim Computer Use is the highlighted first column; the others are the computer-use servers people reach for. each cell comes from that project's own README or docs in September and October 2026 (sources under Credits and license). ✅ present · ❌ absent or not documented · ⚠️ partial.
Capability
Munim Computer Use
Cua Driver
Codex Computer Use
OpenAI Agents API
Anthropic reference demo
Windows-MCP
MacOS-MCP
open-computer-use
computer-use-mcp (zavora)
Notes
macOS
✅
✅
✅
n/a
❌
❌
✅
✅
✅
The Anthropic demo drives a Linux desktop inside Docker, not your machine. The OpenAI Agents API (September 2026) runs a browser OpenAI hosts, so it never drives your machine either.
Windows
✅
✅
✅
n/a
❌
✅
❌
✅
✅
Windows-MCP is Windows only; MacOS-MCP is macOS only.
Linux
✅
✅
❌
n/a
✅ (sandbox)
❌
❌
✅
✅
Munim Computer Use uses AT-SPI + X11; native Wayland apps get element actions but not coordinate clicks.
Accessibility tree with element ids
✅
✅
⚠️
❌
❌
✅
✅
✅
✅
Codex reads the accessibility tree first on macOS and works from screenshots on Windows; the Agents API and the Anthropic demo are screenshot-driven. Ids let the agent press the button instead of a pixel.
Background input (your mouse never moves)
✅
✅
⚠️
n/a
n/a
❌
❌
❌
❌
Munim Computer Use addresses events to the target window: always on macOS (SkyLight and per-process events), for UI Automation patterns and classic Win32 controls on Windows; other Windows UI and all Linux pointer input (XTEST) still move the real pointer. Codex works in the background on macOS with cursors of its own, and takes over the pointer on Windows. Cua Driver delivers in the background by default and falls back to the foreground when that does not land. The other servers in this table drive the real cursor.
Agent pointer overlay
✅
✅
✅
❌
❌
⚠️
❌
❌
❌
Windows-MCP flashes a border around captures; Codex and Cua Driver draw their own cursor on your screen.
Zoom into a region at full resolution
✅
✅
❌
❌
✅
❌
❌
❌
❌
Anthropic's toolset has zoom; here it is a tool on every platform.
Screenshots carry screen-coordinate mapping
✅
⚠️
n/a
n/a
n/a
❌
❌
❌
❌
Origin and pixels-per-point in every capture, so clicks from Retina or downscaled images land.
Hover, wait, label query
✅
⚠️
⚠️
❌
⚠️
✅
⚠️
❌
⚠️
Windows-MCP has Wait/WaitFor; MacOS-MCP has Wait; Anthropic has wait/mouse_move.
Your signed-in Chrome, own tab group or yours
✅
⚠️
⚠️
❌
❌
⚠️
❌
❌
❌
Codex uses its in-app browser; Cua Driver attaches to a browser over CDP; Windows-MCP reads the DOM of open browsers. Munim Computer Use opens its own labelled tab group in your real Chrome, and can also take over a tab you already have open when you ask it to.
Act and observe in one call
✅
✅
n/a
n/a
✅
❌
❌
❌
❌
return_state on any action appends the fresh accessibility tree or page snapshot, so one call acts and checks. Cua Driver's run_steps runs several actions and reports what changed. The Anthropic demo returns a screenshot after each action; Codex and the Agents API run the loop themselves.
Several actions in one call
✅
✅
n/a
n/a
❌
⚠️
❌
❌
✅
Cua Driver and computer-use-mcp (zavora) run a list of steps in one call with run_steps; Windows-MCP's MultiSelect and MultiEdit act on several elements at once. Munim Computer Use's run_steps does the same, checks the whole list before running any of it, and stops at the first failure.
Read a page's text in a signed-in tab
✅
⚠️
❌
❌
❌
⚠️
❌
❌
❌
browser_read returns the page as text with headings, in chunks, with a query filter, from a background tab. Windows-MCP scrapes pages.
Sign-in without the model seeing the password
✅
❌
❌
✅
❌
❌
❌
❌
❌
browser_request_credentials opens a Chrome window naming the site's real origin; what the user types goes into the page and never into the conversation. The Agents API hands sign-in to the developer's app the same way. Desktop password fields refuse typing by default.
Per-app and per-site allow / ask / block rules
✅
❌
⚠️
⚠️
❌
❌
❌
❌
❌
A policy file the user controls; ask shows an approval prompt once per task. Codex asks before each new app and keeps an "Always allow" list; the Agents API asks the developer's app before each new website.
Works with any MCP client
✅
✅
❌
❌
❌
✅
✅
✅
✅
Codex Computer Use is Codex only; the Anthropic demo is Claude only.
Identical tool surface on every platform
✅
⚠️
n/a
n/a
n/a
n/a
n/a
⚠️
✅
33 tools with byte-identical schemas across the Swift and Rust servers. Cua Driver keeps shared parameters in step with a CI check, with a few tools on one platform only.
Prebuilt signed binaries + npm launcher
✅
✅ (script)
✅
n/a
❌
❌
❌
✅ (npm)
✅ (npm)
macOS universal (Developer ID signed) and Windows x64 on Releases.
The chart counts the table above (n/a cells are left out), so it shares the table's bias: we picked the rows.
Benchmarks
Reading a window is the step an agent repeats most. Here every Windows server we could run reads the same File Explorer window as the folder grows from 100 to 20,000 files. All of them read only the rows on screen, so the lines are flat; what differs is the level. munim-computer-use is the fastest at every size and returns the fewest tokens, about half of the next.
Server
Read tool
100 files
1,000
5,000
20,000
Tokens per read (≈)
munim-computer-use 0.8.0
get_app_state
0.14 s
0.13 s
0.16 s
0.14 s
2.0K
Cua Driver 0.38.0
get_window_state, include_screenshot: false
0.20 s
0.18 s
0.18 s
0.19 s
3.9K
Windows-MCP 0.8.5
Snapshot
0.77 s
0.77 s
0.78 s
0.76 s
3.8K
computer-use-mcp 7.8.0 (zavora)
get_ui_tree (COMPUTER_USE_PROFILE=full)
0.43 s
0.50 s
0.43 s
0.52 s
9.3K
open-computer-use 1.1.2
get_app_state
3.95 s
3.82 s
3.70 s
4.23 s
5.9K, with a screenshot
How it was measured, on 27 October 2026: Windows 11 (build 26200), one 1920×1080 display. For each folder size a fresh File Explorer window was opened in front and every server was started once, called over stdio with its own read tool and default arguments (text only where the tool has a switch for it; open-computer-use always attaches a screenshot), one warm-up call dropped, then 5 timed calls; the whole run was done twice, so each time is the median of 10. Tokens are estimates: text at 4 characters per token, images by Anthropic's formula (width × height / 750 after the API's resize). Windows-MCP's Snapshot also lists the desktop's other windows; the others read one window. Terminator was left out: its get_window_tree picks the first explorer process, which is the taskbar, and it takes no pid.
Acting, on the same PC: the five clicks of 1 + 2 = on Calculator (clear, 1, +, 2, =), each a separate call by element id, after one read. Median of 3 runs.
Server
Five clicks
Per click
munim-computer-use 0.8.0
1.70 s
0.34 s
Cua Driver 0.38.0
2.32 s
0.46 s
munim-computer-use 0.7.3
2.98 s
0.60 s
Most of a click is the agent pointer flying to the target so you can see where the agent acts; 0.8.0 halves that flight. run_steps sends all five in one call.
The same window read by every Windows release, from the release binaries, in two rounds of all 17 with Windows-MCP in each round, plus 0.8.0 measured against 0.7.3 the same way (a separate run from the one above, so the times differ a little). 0.7.0 is where get_app_state started reading only what is on screen and fetching each element's attributes in one request; 0.8.0 fetches the whole window in one request instead of one per element, with the same output.
Release
Read time (median)
Characters
Tokens (≈)
0.2.0
0.40 s
9,860
2.5K
0.3.0
0.43 s
9,860
2.5K
0.3.1
0.38 s
9,860
2.5K
0.4.0
0.38 s
9,860
2.5K
0.4.1
0.38 s
9,860
2.5K
0.4.2
0.39 s
9,860
2.5K
0.4.3
0.40 s
9,860
2.5K
0.4.4
0.38 s
9,860
2.5K
0.5.0
0.38 s
9,860
2.5K
0.5.1
0.37 s
9,860
2.5K
0.5.2
0.43 s
9,860
2.5K
0.6.0
0.37 s
9,860
2.5K
0.6.1
0.41 s
9,860
2.5K
0.7.0
0.23 s
7,499
1.9K
0.7.1
0.28 s
7,499
1.9K
0.7.2
0.25 s
7,520
1.9K
0.7.3
0.25 s
7,520
1.9K
0.8.0
0.16 s
7,515
1.9K
Windows-MCP 0.8.5 Snapshot
0.75 s
14,929
3.7K
node scripts/readme-charts.mjs redraws the charts; the coverage chart is counted from the capability matrix.
Why it works well
Accessibility first, pixels second.get_app_state returns the app's accessibility tree with stable element ids, so the agent presses the button instead of guessing at a coordinate. It costs a fraction of the tokens of a screenshot and it is what scores highest on OSWorld-style tasks. Screenshots are for verifying and for content the tree cannot describe.
What is on screen, fast.get_app_state lists what is in view: a list or table gives its visible rows, and anything scrolled away is skipped and counted (offscreen: true lists it too). Attributes are read in one request per element, and on Windows the whole window comes back in a single request. A Finder window of 2,000 files reads in 165 ms instead of 2.2 s, and File Explorer in about 0.15 s, faster than any other Windows server we measured (see Benchmarks).
Dialogs first. While a sheet, alert or popover blocks a window, only its controls are listed, with a note saying so, so the agent answers the dialog instead of clicking behind it.
Electron and Chromium apps (macOS). Slack, VS Code, Discord and other Chromium-based apps expose their tree only to a client that asks; get_app_state asks, and says so while a background app is still building it.
Text the tree cannot describe. When an app exposes no labelled controls, its window is read with on-device OCR (Apple Vision on macOS, Windows OCR on Windows) and the text is listed with ids that click, hover, right_click and drag accept. The image never leaves the machine. ocr: "always" or "never" overrides it.
Ids that fail instead of misfiring. Every id is rechecked before an action. If its element is gone, or its role or name changed (a dialog swapped its buttons, a list re-sorted), the action fails with a clear error instead of acting on whatever sits there now.
Waits as long as the app takes.return_state waits for the app's accessibility notifications to stop instead of a fixed pause: about 230 ms for a field edit (was 350 ms), and long enough for a sheet to finish sliding in.
Background control. Events are addressed to the target window (SkyLight on macOS, UI Automation patterns and posted window messages on Windows). On macOS the agent never takes your mouse or keyboard; see Works alongside you for the exact guarantee on each platform.
Pointer overlay, not your pointer. A soft lavender agent pointer shows where the agent is acting. Your cursor is untouched.
Coordinates that land. Every screenshot and zoom carries its screen origin and pixels-per-point. zoom captures any region at full physical resolution.
Your browser, your logins. The Chrome extension gives the agent its own labelled tab group in your signed-in Chrome, and leaves your tabs alone unless you point it at one.
Or the tab you already have open.browser_list_tabs all=true shows every tab in the browser and browser_use_tab takes one over in place — useful when the page is already signed in or mid-flow and re-opening the URL would throw that away. An adopted tab is not moved into the agent's group, not activated and not reloaded; cleanup releases it rather than closing it, and browser_release_tab hands it back early.
Tabs the agent's clicks open are the agent's too. A target=_blank link or popup opened by an agent click or key press within 10 seconds joins that task's tabs and group, and its cleanup closes it. A tab opened from an agent tab without a recent agent action stays yours.
Parallel tasks, one extension. Every MCP process gets its own tab group, and one process can run several tasks by passing a stable session_id on its browser calls. A task cannot drive or adopt another task's tabs, and cleanup (including a process exiting) closes only its own. Any number of MCP processes share the one extension: the first owns it and the rest go through it, and if the owner exits another takes over without closing anyone's tabs. Tasks share Chrome's cookies and logins, and desktop apps and the clipboard are not isolated.
Model-agnostic. No vision model is required for interaction; local models work too.
Look → act → verify.hover for mouse-over menus, wait for loads, query to find a control by label without reading a whole tree.
Act and look in one call. Pass return_state: true to any action (click, type_text, set_value, browser_click, browser_navigate, …) and the result carries the app's fresh accessibility tree, or the page's current snapshot. Browser actions report readiness separately: a loaded tab is not proof that delayed SPA UI has rendered. Use wait_for_selector for an expected visible element (see below). state_query narrows the desktop tree the same way query does.
Several steps, one call.run_steps takes a list like click the field, type, press return and runs it in order, each step with the same checks as a direct call. The whole list is checked before anything runs, the first failing step stops the rest, and the result says what ran and what did not. Put return_state on it to see the app after the last step.
Pages as text.browser_read returns a page's readable text with its headings, including what is scrolled out of view, from a tab in the background. It comes in chunks you can continue with offset, query keeps only the matching lines under their heading, and include_links lists the links.
Passwords stay out of the conversation.browser_request_credentials asks the user to sign in in a small Chrome window that shows the site's real origin. What they type goes straight into the page's fields and is never returned to the model. browser_snapshot never shows a password field's value, and on the desktop, typing into password fields is refused by default.
Pick up what you are looking at.get_app_state and screenshot take app: "frontmost" for the app in front of the user, so a client can hand the agent "this window" in one step.
Works alongside you
The agent has its own pointer; yours stays yours.
macOS — guaranteed. Every action goes through accessibility (press, set value, select text, show menu, scroll bars) or through events addressed to the target app's process and window. The server never moves your pointer, never posts into the system-wide input stream, and never holds or blocks your input, so you can keep clicking and typing in other apps while the agent works — even in the same app, on another window. Events come from a private source, so a modifier you are holding does not leak into the agent's clicks. The target app may be brought forward when that is the point of the step (activate_app, or handing you a password field), but not on every action.
Command shortcuts sent to a background app run through its menu bar (macOS gives menu key equivalents only to the front app), so cmd+W, cmd+S and the like work without bringing it forward. A minimized window, or a hidden app, is moved for the moment it needs pixels or input (typing, a coordinate click, a screenshot, OCR) onto an invisible display, where it works while nobody sees it; it is minimized or hidden again after 20 idle seconds, when another app's window needs the display, or when the server exits. If you bring the app up yourself meanwhile, its windows come back on screen where they were. Accessibility actions on such a window (press, set value) need none of this. COMPUTER_USE_PARK_WINDOWS=0 turns it off, and those actions then return an error asking for the window to be restored.
The exceptions refuse instead of borrowing your pointer: a click, hover or scroll with no target app (coordinates over the desktop before any get_app_state), and drags that leave the source window (between apps, or onto the desktop). The error says what to pass instead.
Windows — best effort, reported. Element presses (Invoke), set_value, select_text and scrolling through UI Automation's ScrollPattern never touch your pointer, and classic Win32 controls also take clicks, hovers, wheel and drags as posted window messages. Other UI (Chromium, Electron, WPF, UWP) only reacts to real mouse input, so those clicks, hovers and drags move your pointer, and type_text/press_key go to the focused window. Whenever the real pointer was used, the result says via cursor. A coordinate action on an element that another app's window covers is refused rather than clicking that window, and pointer and keyboard actions on a minimized window return an error asking for it to be restored (accessibility actions still work on it).
Linux — pointer actions use it. XTEST input moves the real pointer and goes to the focused window, and results say via cursor. Element actions through AT-SPI (press, set value, insert text, select) do not move it.
Apps and sites the agent may use
You decide which apps and websites the agent may touch. Put a policy.json in the server's support directory (~/Library/Application Support/computer-use on macOS, %LOCALAPPDATA%\munim-computer-use on Windows, ~/.local/share/munim-computer-use on Linux; munim-computer-use identity prints it as supportDir), or point COMPUTER_USE_POLICY at a file anywhere. An app that embeds the server under its own profile reads the file from its own support directory:
allow, ask or block per app or site. ask shows the user a prompt the first time the agent reaches for it (a system dialog for apps, a small Chrome window for sites), and an approval lasts until that agent task ends. An unanswered prompt counts as no after two minutes.
Apps match their name or bundle id, ignoring case. The rule applies when the agent reads, captures or activates the app. Actions then target elements from a snapshot that was allowed.
Sites match a host and all its subdomains, and the most specific pattern wins. They are checked on every browser action against the page the tab is showing at that moment, so following a link into a blocked site does not get around it.
* sets the default, which is otherwise allow, so {"apps": {"*": "block", "Notes": "allow"}} is an allow-list.
Changes apply at once, with no restart. A file that is not valid blocks everything rather than being ignored, so a typo cannot quietly turn a block into an allow.
This is a guard rail for an agent that follows its instructions, not a sandbox. A coordinate click lands wherever it points, and a whole-display screenshot shows every window.
Every action tool also takes return_state, which returns current state after the action in the same call. run_steps runs a list of them in one call and stops at the first that fails.
Browser readiness and safe re-observation
browser_click, browser_type, browser_press_key, and browser_navigate accept
wait_for_selector together with return_state: true. browser_snapshot accepts
the same wait without performing an action:
The extension polls for any matching visible element in the top-level document,
independently of resource loading, including same-document rendering. It shares
the snapshot's visibility rules: at least 2×2 CSS pixels; no hidden, aria-hidden,
inert or fully transparent ancestor. Self-transparent native form controls remain
available for styled upload/checkbox hit targets; transparent containers do not.
This is not a guarantee of clickability or
lack of occlusion. Frames and shadow
roots are not searched. Choose a selector specific to the new expected UI;
an element already present can satisfy it immediately. No arbitrary page script
is accepted, and a selector may not test the value attribute: React mirrors a
controlled input's value there, password fields included, so a wait could
otherwise confirm a password one guessed character at a time.
wait_timeout_ms: defaults to 8000, accepts integers 0–10000; 0 checks
once. Requires wait_for_selector. The budget covers polling, not browser/API
latency or user site-approval prompts. Site rules are rechecked before every
observation and before the final snapshot.
readiness: met (visible_selector, … ms): the specified condition was observed,
not a guarantee that all network requests, animations, or application work ended.
readiness: timeout: the condition was not observed within the budget. The
action's success remains separate, and the current snapshot is still returned.
readiness: error: the condition could not be evaluated (for example invalid
CSS); the action can still have succeeded. Correct the wait and re-observe.
Without a selector, return_state retains the short pause and bounded
navigation-load wait, but reports not_requested (document_load, … ms) rather
than claiming SPA readiness. A navigation-load budget expiry reports timeout.
A plain browser_snapshot without a wait remains an immediate observation.
If a returned snapshot looks unchanged, do not repeat a non-idempotent click
just because expected UI is absent. Wait/re-observe first:
Send this to browser_snapshot, inspect its readiness and fresh indices, then
choose the next action. A timeout or tool error does not roll back a click,
submission, typing, or navigation. Use current state to decide whether a retry is
safe. Invalid wait types/bounds or an action wait without return_state: true are
rejected before acting. null for an optional argument means the same as leaving
it out. These options require an updated extension as well as the native server;
when an older extension ignores them, the result says so and asks for a reload.
Regression checks: node --test chrome-extension/background.test.mjs covers the
wait contract and site-policy guards. The real-browser CI harness
scripts/e2e-browser.mjs tests a dialog delayed by 1100 ms without navigation,
one click only, plus timeout and read-only recovery. The dependency-free CDP
regressions cover both snapshot scoping/pagination and SPA readiness. Run against
an isolated Chrome/Chromium profile; no Playwright module is required:
node scripts/snapshot-dom.test.mjs --chrome /path/to/chrome
# Extract the pre-fix source; each suite must fail independently:
git show deb4c7a:chrome-extension/background.js > original-background.js
node scripts/snapshot-dom.test.mjs --chrome /path/to/chrome --source original-background.js --suite snapshot
node scripts/snapshot-dom.test.mjs --chrome /path/to/chrome --source original-background.js --suite readiness
browser_snapshot automatically scopes to the topmost visible dialog when one is
open, so covered background controls cannot consume the dialog's budget. A
dialog's portalled popups stay in scope: elements its controls point at through
aria-controls/aria-owns, open popovers, and controls painted on top of the
dialog, such as a select's options rendered at the end of <body>. Hidden
and inert subtrees are excluded; reachable offscreen controls remain available.
Native modeless dialogs and explicit aria-modal="false" do not scope the page.
Native modal dialogs and explicit aria-modal="true" are definitive candidates.
Implicit ARIA dialogs are heuristically scoped only when they cover the viewport
centre and are topmost there, rather than merely at their own centre. With no
hit-test stack, centre coverage is the fallback. Corner chat widgets and bottom
cookie banners therefore retain the page's indexed controls.
It returns up to limit controls (default and maximum 250), reports scope,
offset, limit, and total, and explicitly says when it is truncated and which
offset to request next. Continue on the same tab_id and session_id. Indices
are global within the selected scope and stay actionable across pages of an
unchanged UI; after a UI change or navigation, take a fresh snapshot from
offset=0. An action's return_state also starts at offset=0 and uses the same
scope and truncation reporting.
Names, argument shapes and descriptions are identical on every platform, and CI enforces it (node scripts/check-tool-parity.mjs); a model that learned them on a Mac needs nothing new on Windows. To change a tool, edit both literals at once with scripts/tool-defs.mjs rather than by hand.
Repository layout
Directory
What
Build
macos/
Swift server on the Accessibility API and ScreenCaptureKit (macOS 14+)
swift build -c release → .build/release/munim-computer-use
windows-linux/
Rust server: UI Automation on Windows, AT-SPI + X11 on Linux
These are needed by the screen-capture dependencies even on X11. In particular,
a missing libpipewire-0.3.pc during the build means pipewire-devel is missing.
# macOS
cd macos && swift build -c release
# Windows / Linux
cd windows-linux && cargo build --release
Linux notes: element actions work everywhere; coordinate clicks need an X11 or XWayland client, since native Wayland apps do not expose absolute geometry.
Tests
cd windows-linux && cargo test # Rust server
node chrome-extension/background.test.mjs # extension, against a fake Chrome
node scripts/check-tool-parity.mjs # Swift and Rust tool lists match
# Snapshot and SPA readiness regressions (dependency-free CDP, isolated profile):
node scripts/snapshot-dom.test.mjs --chrome <Chrome for Testing binary>
# The browser tools end to end, in a throwaway Chrome for Testing profile that
# never touches your own Chrome (npx playwright install chromium to get one):
node scripts/e2e-browser.mjs --server <munim-computer-use binary> --chrome <Chrome for Testing binary>
Register the native messaging host: munim-computer-use install-native-host (for example npx -y munim-computer-use install-native-host), or from a checkout sh chrome-extension/install.sh (macOS/Linux) / powershell -File chrome-extension/install.ps1 (Windows), which find the build and run the same command. Point COMPUTER_USE_PATH at the binary if it is not in the default build location.
If the extension still gets Specified native messaging host not found., or the browser tools say the extension is not connected, run munim-computer-use check-native-host in an ordinary terminal. It looks the host up the way Chrome does (on Windows: registry views and policies; everywhere: the manifest's encoding, name, type and allowed origins), starts it with Chrome's command line, and names the step that would fail. Two Windows causes look fine from the inside. An installer run by an MSIX-packaged app (some agent desktop apps are packaged) writes to a private, virtualised copy of HKCU and %LOCALAPPDATA% that Chrome never reads. One run by a sandbox with its own Windows account writes to that account's HKCU. The check detects both; run install-native-host again from your own terminal. install-native-host reports the same findings under problems.
The standalone server's host is com.munimtech.computer_use.desktop; MT Code, which bundles this server, registers com.munim.mtcode.desktop. The extension connects to every host it knows at once and answers each on its own connection, so MT Code and a standalone server (npx, Claude Code, Cursor, a checkout) can both drive Chrome at the same time, each in its own tab groups.
Sandboxed agents (Windows)
Some agents run their commands in a sandbox: a restricted or low-integrity token, a separate Windows account, or another desktop. A server started there can still list windows and their titles, but other apps' UI Automation refuses it and captures come back black, without any error. When that happens the server says so: get_app_state explains an empty tree, screenshot and zoom flag a capture that is one flat colour, and activate_app checks that the window really came forward. Each names the cause it found (for example "runs at low integrity" or "runs as another account"). Retrying from the same context will not help; configure the agent to start the MCP server in your own, unsandboxed context.
Environment flags
Variable
Effect
COMPUTER_USE_BROWSER=0
Hide the browser_* tools
COMPUTER_USE_AGENT_CURSOR=0
Do not draw the agent pointer
COMPUTER_USE_AGENT_CURSOR_TASK_FADE_SECS
How long the pointer stays after the last tool call (default 8)
COMPUTER_USE_ALLOW_SECURE_FIELD_INPUT=1
Allow typing into password fields (refused by default)
COMPUTER_USE_REMOTE_CONTROL=1
Remote-desktop mode: input takes over the real pointer
COMPUTER_USE_PARK_WINDOWS=0
macOS: do not move minimized or hidden windows to an invisible display to act on them
COMPUTER_USE_POLICY=<file>
Where to read the app and site policy (default policy.json in the support directory)
Remote control
Normally this server never touches the pointer: coordinate clicks are routed to
a specific window, keystrokes are posted to a specific process, and an action
that cannot be targeted is refused rather than taking over the machine. That is
what lets an agent work while the user keeps using their computer.
COMPUTER_USE_REMOTE_CONTROL=1 inverts that contract for one process, for the
case where a person is watching this machine's screen from another one and is
steering it themselves. Then click, right_click, drag, hover and
scroll move the real cursor and type_text and press_key go to whatever is
focused, the way Chrome Remote Desktop or Screen Sharing behave. scroll also
accepts x/y so the wheel acts over the point the viewer scrolled at, and the
agent-cursor overlay stays hidden — there is only one pointer now.
A host that runs both an agent and a viewer runs them as two processes, so
turning this on for the viewer never takes the pointer away from the user on
the agent's behalf.
Embedding in an app
An app can ship this binary inside its own bundle and run it under its own identity, so it never shares a browser bridge, agent-cursor app or native-messaging host with a standalone install on the same machine. MT Code does exactly this. Pass a profile, either as a JSON object or a path to a JSON file, with --profile <json|file> (any position) or COMPUTER_USE_PROFILE:
What install-native-host registers, and for which extension. The first name is this identity's own; later ones are aliases, written only if no other installed app owns them
Each of supportDir, bridgeSocket, envPrefix, agentCursorName, agentCursorBundleId and historyDir can also be overridden by COMPUTER_USE_<SNAKE_CASE> (for example COMPUTER_USE_SUPPORT_DIR), which wins over the profile. munim-computer-use identity prints the resolved values.
Browser bridge. Run munim-computer-use install-native-host with the same profile. It writes a wrapper that relays Chrome into this identity's bridge (replaying the profile), and a host manifest for each name in every Chrome/Chromium profile directory (the registry on Windows). It rewrites nothing that is already current, so an app can call it on every launch.
Extension. Use the stock extension (it connects to com.munim.mtcode.desktop and com.munimtech.computer_use.desktop), or build a variant with its own host names, tab-group title and key: node scripts/build-extension.mjs --out <dir> --host com.example.desktop --group-title "Example" --key <base64>. It prints the variant's extension id for extensionIds.
macOS permissions. The MCP server is a bare executable, so Accessibility and Screen Recording are granted to the app that spawns it. Only the agent-cursor overlay has a bundle of its own; ship <agentCursorName>.app (a copy of the binary plus an LSUIElement Info.plist) beside the binary, or it is materialised under supportDir on first use.
Prompting your agent
Look → act → verify. get_app_state for ids, act by id with return_state: true to see the result in the same call, and screenshot when the tree cannot show it. Read pages with browser_read, and let the user type passwords through browser_request_credentials. Use zoom for small text, hover for menus that appear on mouse-over, wait after loads, keyboard shortcuts for stubborn widgets. The system-prompt text MT Code gives its agents lives in CodexDeveloperInstructions.ts and is a good starting point.
Contributing
The server is developed here. MT Code ships it by pinning a release of this repository (native/munim-computer-use.json), so changes reach MT Code with the next release. Issues and pull requests are welcome.
Credits and license
Designed and built by Munim Technologies (Munim, Inc.) for MT Code. On-screen pruning, dialog scoping, notification-based settling, OCR fallback, menu shortcuts and window parking follow arc-cua (MIT). Copyright 2026 Munim, Inc. Licensed under the Apache License 2.0; see LICENSE.
Open-source Computer Use MCP server for any coding agent — macOS, Windows, Linux. Accessibility-first, background input, agent pointer, zoom, your signed-in Chrome.
上次检查时间：2026-11-09。 Open-source Computer Use MCP server for any coding agent — macOS, Windows, Linux. Accessibility-first, background input, agent pointer, zoom, your signed-in Chrome.