Just Another OpenCode Computer Use
Desktop control MCP server for AI agents. Gives opencode, Claude Code, Cursor, and any MCP-compatible client the ability to control desktop applications and web browsers on the host machine. Built on OS-native accessibility APIs with a CDP browser bridge, all wrapped in a safety sandbox.
Features
- 🖥 Desktop Control — mouse click, move, drag, scroll, type text, press keys
- 📸 Screen Capture — capture display as PNG for vision analysis
- ♿ Accessibility Tree — structured UI tree via OS-native accessibility APIs (deterministic, zero vision cost)
- 🔍 Element Finding — locate UI elements by role, label, or text
- 🌐 Browser Bridge — CDP-based Chrome/Firefox control (DOM, JS, tabs)
- 🛡 Safety Sandbox — confirmation gates, filesystem sandbox, network allowlisting
- 🖼 Vision Fallback — optional VLM (Ollama/OpenAI) for screenshot-based element detection
Platform Support & Requirements
| Platform | Tools Used | Status |
|---|---|---|---|
| Linux (X11/Wayland) | xdotool, wmctrl, scrot/grim/import, xclip/xsel/wl-clipboard | ✅ Tested |
| macOS 12+ | osascript, screencapture, pbpaste, pbcopy (all built-in) | 🟡 Tested (community) |
| Windows 10+ | powershell, cmd.exe, .NET Framework (built-in) | 🟡 Designed (untested) |
Permissions & Setup by OS
macOS
macOS uses built-in CLI tools — no package install needed. However, two system permissions must be granted:
Important: Grant these permissions to the app that runs the
ocubinary (e.g. Terminal, iTerm2, Warp, VS Code). Ifocuruns as an MCP subprocess, you may need to grant permissions to the host app (e.g. Terminal if you launched opencode from it).
Verification:
# Check if osascript can list windows (requires Accessibility)
osascript -e 'tell app "System Events" to get name of every window of every process'
# Check if screencapture works (requires Screen Recording)
screencapture -x /tmp/test-capture.png && open /tmp/test-capture.png
Setup:
# All tools are built-in — no packages to install
./scripts/setup.sh
Linux (X11)
Linux uses external CLI tools that must be installed via your package manager:
Wayland alternatives:
Permissions:
- No OS-level permission grants needed
xdotoolneeds access to the X11 display ($DISPLAYmust be set, typically:0)- On Wayland,
xdotool/wmctrlmay not work — use the Wayland alternatives above
Setup:
./scripts/setup.sh # auto-detects display server and installs appropriate tools
Windows
Windows uses built-in tools — no package install needed.
Permissions:
- No special OS-level permissions required for default operation
- PowerShell execution policy must allow script execution (typically already set)
- Some operations (e.g. window enumeration via
user32.dllP/Invoke) are always allowed
Setup:
./scripts/setup.sh # verifies PowerShell availability
Quick Start
1. Install Dependencies & Permissions
Follow the per-OS guide above to install tools (Linux) or grant permissions (macOS), then:
./scripts/setup.sh
2. Build
cargo build --release
cp target/release/ocu ~/.local/bin/
3. Run
ocu --help
ocu # stdio transport (default)
ocu --transport sse --port 8080 # SSE transport
ocu --config ~/.config/opencode/ocu.json # with config file
4. Add to opencode.json (opencode)
{
"mcpServers": {
"computer-use": {
"command": "/home/YOU/.local/bin/ocu",
"args": ["--config", "/home/YOU/.config/opencode/ocu.json"]
}
}
}
5. Add to opencode.json (Claude Code / Cursor / other MCP clients)
{
"mcp": {
"computer-use": {
"type": "local",
"enabled": true,
"command": "/home/YOU/.local/bin/ocu",
"args": ["--config", "/home/YOU/.config/opencode/ocu.json"],
"env": {}
}
}
}
MCP Tools
Computer
Accessibility
System & Browser (planned)
Architecture
ocu binary (Rust)
├── MCP Server (rmcp)
├── Tool implementations
├── Platform abstraction traits
└── Platform backends
├── macOS (AX, CGEvent, CoreGraphics)
├── Windows (UIA, SendInput, DXGI)
└── Linux (AT-SPI2, XTest, X11/Wayland)
Design Philosophy
- Accessibility-first — uses OS-native a11y APIs for deterministic, token-efficient UI interaction
- Vision fallback — screenshots + VLM for cases where a11y is unavailable
- Zero runtime deps — single statically-linked binary
- MCP-native — works with any MCP client out of the box
- Safe by default — sandbox, confirmation gates, audit log
License
MIT