Potencialize seu agente
Instalação do publicador. Não é um endpoint do PlugMCP.
Host a copy with our keys and logs.
Ainda não há uma versão empacotada no registro. Siga o README do repositório.
URL remota ou um comando na sua máquina? Como as duas instalações diferem
Rodar no PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
AbrirDo publicador
A Model Context Protocol (MCP) server that gives AI agents human-like control over Windows. Combines visual perception (screenshots) with simulated mouse and keyboard input to operate any application exactly the way a person would — no API integration required.
perceive, describe_screen) for routine checks (~1k tokens), fall back to full screenshots only when visual confirmation is needed (~5-50k tokens), and call native APIs directly when available for zero-token operations.This project was built by someone who doesn't know how to code.
The author is a Genshin Impact player who wanted AI to help with repetitive in-game tasks — farming, map clearing, daily commissions. The goal was simple: let AI see the screen and operate the game just like a human would, reducing the grind without risking account bans from API-based automation.
The entire project was developed using Claude Code + a local Gemma 4 26B Q4 quantized model running on a single RTX 5060 Ti 16G — the best hardware the author could access. No cloud compute, no engineering team, no prior programming experience.
The author estimated cloud API costs would be too expensive for the volume of iteration needed, so everything was done locally. The tradeoff: a small local model that frequently made mistakes, wrote broken code, and struggled to maintain context across restarts.
Despite not understanding code syntax, the author discovered a workflow that made development possible:
"I don't understand code, but I know my project's logic inside and out. When something breaks, I know exactly where to look. I became an AI error corrector — the AI writes, I review, the AI fixes. That loop works even if you've never written a line of code."
From the first npm install to this release: about one month. The author learned what Node.js is, what Rust does, how MCP works, and what "compilation" means — all through conversations with Claude Code. Every concept in this project was explained by AI to someone hearing it for the first time.
The author notes that using a cloud model (like the Claude Opus powering this session) would have made development significantly easier — faster iteration, fewer mistakes, better code generation. But the constraint of a small local model forced a deep understanding of the project's logic, which turned out to be the real skill that made this possible.
The current version works, but the author's vision goes further: integrating specialized Agents and purpose-built models to handle complex multi-step tasks with better reliability. The end goal is an AI that can truly play games alongside humans — not as a bot, but as a collaborative agent that sees and operates the screen like a person.
Key takeaway: You don't need to be a programmer to build software with AI. You need clear thinking, a logical chain, and the willingness to be the AI's "error corrector."
npm install -g @betrayzl/windows-computer-use-mcp
This installs a pre-built package — no Rust compiler needed. After installation, configure your MCP client to launch:
node C:\Users\<你的用户名>\AppData\Roaming\npm\node_modules\@betrayzl\windows-computer-use-mcp\bundle\index.js
Or find the exact path with:
npm root -g
windows-computer-use-mcp-v*.zipstart.bat, or run node bundle/index.jsNo Rust compiler required.
Requires Rust stable toolchain in addition to Node.js.
git clone https://github.com/betrayzl/windows-computer-use-mcp.git
cd windows-computer-use-mcp
npm install
node build.js
The build script performs four stages:
napi build --platform --release in native/tsc compiles src/ → dist/@vercel/ncc bundles dist/index.js into a single bundle/index.js.node file is copied to bundle/win-cu-native.nodeA successful build outputs:
✨ Build complete!
📦 Output location: bundle/
👉 You can now configure your MCP client to use: node bundle/index.js
npm install -g @napi-rs/cli
Then run node build.js again.
Ensure you ran node build.js (not just tsc) and the Rust toolchain is installed. The build script compiles native/ from source — the .node file must match your CPU architecture.
# Verify Rust is installed
rustc --version
cargo --version
This can happen when running in a virtual machine or via RDP without a GPU. Try using describe_screen or perceive instead, which use UI Automation (no GPU required).
The target window may be running as Administrator while the MCP server is not. Run the MCP server with the same privilege level as the target application.
Add to your MCP client configuration (Claude Desktop, OPENCLAW, etc.):
{
"mcpServers": {
"windows-computer-use": {
"command": "node",
"args": ["path/to/windows-computer-use-mcp/bundle/index.js"]
}
}
}
You can also run it directly for testing:
node bundle/index.js
# MCP server now listening on stdio
Here is a typical session using 4 tools to accomplish a task:
1. perceive({ targetProcess: "notepad" })
→ Check what's on screen, whether notepad is in foreground
→ Returns: elements list, foreground info, occlusion warning
2. focus_app({ processName: "notepad" })
→ Bring notepad to front if it was obscured
3. click({ x: 500, y: 300, processName: "notepad" })
→ Click the text area (processName ensures notepad is frontmost first)
4. type({ text: "Hello from MCP!" })
→ Type text into the focused editor
5. screenshot({})
→ Visually confirm the result (with a vision-capable model)
When your AI model supports image inputs, you get the full "human-like" experience:
screenshot to see the entire screenclick, type, drag to interactWhen your model cannot process images, use the cost-efficient perception tools:
perceive (~1k tokens) to get a structured description of UI elementsdescribe_screen to get a text summary of what's visibleget_ui_elements for detailed element trees with coordinatesThe most common failure mode in desktop automation is clicking the wrong window. The server provides three safeguards:
processName to click() — automatically focuses the target window before clickingperceive() warning field — detects if your target is obscuredfocus_app() explicitly — brings any window to the foreground// Safe click — always lands on the right window
click({ x: 400, y: 200, processName: "chrome" })
// Before desktop operations
show_desktop()
get_desktop_icons() // auto-calls show_desktop internally
AI Agent (MCP Client)
↕ MCP Protocol (JSON-RPC over stdio)
Windows Computer Use MCP Server
├── TypeScript Layer (src/)
│ ├── index.ts — 24 MCP tool definitions & request handlers
│ ├── executor.ts — High-level API, DPI scaling, focus enforcement
│ ├── utils.ts — Multi-monitor logical↔physical coordinate mapping
│ └── types.ts — TypeScript interfaces
│
└── Rust Native Module (native/src/)
├── capture.rs — DXGI hardware-accelerated screen capture (full + region)
├── input.rs — Keyboard/mouse simulation via enigo
├── uia.rs — UI Automation element tree (41 control types, low-cost perception)
├── window.rs — Window management, 6-level focus strategy, desktop icon arrangement
└── apps.rs — Installed application enumeration
All tools accept and return global logical coordinates (DPI-aware):
We welcome contributions! Here's how to get started:
feat:, fix:, docs:, etc.)git clone https://github.com/betrayzl/windows-computer-use-mcp.git
cd windows-computer-use-mcp
npm install
node build.js # Full build
node bundle/index.js # Test run
node tests/test_driver.js # Run diagnostics
src/index.tsCallToolRequestSchemasrc/executor.tsnative/src/ if low-level Windows API is needednative/src/lib.rsThis server has deep Windows system access (screen capture, keyboard/mouse simulation, process management, cross-process memory). Please review SECURITY.md before use.
This project is licensed under the MIT License — see LICENSE for the full text.
Built with NAPI-RS, enigo, and the Model Context Protocol SDK.
Human-like Windows control for AI agents via the Model Context Protocol. Combines DXGI visual perception with simulated mouse/keyboard input — works on any Windows application without API integration.
Visão geral
Human-like Windows control for AI agents via the Model Context Protocol. Combines DXGI visual perception with simulated mouse/keyboard input — works on any Windows application without API integration. Built by a non-programmer with Claude Code.
Ferramentas
Ainda não há lista de ferramentas. Servidores remotos recebem uma após a primeira verificação de saúde.
Perguntas frequentes
Copie nesta página a configuração stdio para Cursor, Claude Desktop, Claude Code ou VS Code. Ela usa a instalação do publicador, não um proxy do PlugMCP.
Este registro não lista nenhuma variável de ambiente obrigatória.
Windows Computer Use roda localmente via stdio.
Última verificação: 2026-11-05. Human-like Windows control for AI agents via the Model Context Protocol. Combines DXGI visual perception with simulated mouse/keyboard input — works on any Windows application without API integration. Built by a non-programmer with Claude Code.