What if your AI could do everything you do on a computer?
Not just write code โ but open a browser, click buttons, fill forms, run servers, test in real browsers, install anything, and see the screen?
taw-computer is an open-source MCP server that gives AI agents a full Ubuntu desktop inside Docker. Your AI connects, gets a real computer, and works like a human would.
No internal LLM. No chat UI. Your AI is the brain. This is the body.
Demo
Why taw-computer?
Other tools let AI write code. taw-computer lets AI use a computer.
Quick start
Get running in under 5 minutes:
# 1. Clone & build
git clone https://github.com/the-agents-work/taw-computer.git
cd taw-computer
docker build -f images/Dockerfile.taw -t taw-computer-base .
# 2. Install & start
npm install && npm start
Then add to your AI client:
{
"mcpServers": {
"taw-computer": {
"command": "npx",
"args": ["tsx", "/path/to/taw-computer/mcp/index.ts"]
}
}
}
Add to Cursor MCP settings (Settings โ MCP Servers) โ same JSON format as above.
Add to claude_desktop_config.json โ same JSON format as above.
taw-computer speaks standard MCP over stdio. Any client that supports MCP can connect.
Got a powerful server / Mac Mini / VPS? Run taw-computer there and connect from anywhere:
{
"mcpServers": {
"taw-computer": {
"command": "ssh",
"args": ["user@your-server", "cd /path/to/taw-computer && npx tsx mcp/index.ts"]
}
}
}
Your laptop (Claude Code)
โ SSH (stdin/stdout piped over network)
Remote server (taw-computer + Docker)
โ Docker
Ubuntu sandbox
Setup:
- On the server: install Docker, clone repo, build image,
npm install - On the server: enable SSH (
sudo systemctl enable ssh) - On your laptop:
ssh-copy-id user@your-server(passwordless login) - Add the MCP config above โ done!
Watch via VNC: open http://your-server:6080 in your browser.
That's it. Now tell your AI: "Create a VM and build me a website" โ and watch it work.
What can it do?
๐ฅ๏ธ "Build me a landing page"
AI creates a VM โ scaffolds Next.js โ writes components โ starts dev server โ opens browser to check โ iterates until it looks right
๐ "Go to Amazon and find the best laptop under $1000"
AI opens Chromium โ navigates to Amazon โ searches โ scrolls โ extracts prices โ compares โ reports back
๐งช "Run E2E tests on my deployed app"
AI launches Playwright โ navigates to your URL โ fills forms โ clicks buttons โ asserts results โ reports failures
๐ง "Set up a PostgreSQL database with sample data"
AI runs apt install postgresql โ creates database โ writes seed script โ runs it โ verifies with queries
๐ธ "What does my app look like on mobile?"
AI takes desktop screenshot โ resizes viewport โ screenshots again โ compares โ suggests CSS fixes
How it works
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Your AI Client โ
โ Claude Code ยท Cursor ยท Claude Desktop ยท any MCP โ
โโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ MCP protocol (stdio)
โโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ taw-computer MCP server 30+ tools โ
โ vm ยท shell ยท files ยท browser ยท desktop ยท search โ
โโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Docker API
โโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Ubuntu 22.04 Sandbox isolated containerโ
โ โ
โ bash Chromium + CDP xfce4 Desktop + VNC โ
โ git npm pip curl Playwright โ
โ python node xdotool scrot โ
โ โ
โ /workspace โ your project files live here โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
30+ tools
Set-of-Mark: how browser automation actually works
Most "computer use" tools guess pixel coordinates. We use Set-of-Mark prompting โ the AI sees numbered badges on every interactive element:
Step 1: browser_snapshot
โ AI sees screenshot with [1] Login [2] Search [3] Cart ...
Step 2: browser_click_ref(ref=2)
โ clicks the Search box precisely
Step 3: browser_type_ref(ref=2, text="laptop", submit=true)
โ types and presses Enter
Step 4: browser_snapshot
โ sees new page with results [4] [5] [6] ...
No coordinate guessing. No CSS selector fragility. The AI sees what it's clicking.
VNC โ watch your AI work in real time
Every sandbox comes with a noVNC web viewer. Open the URL in your browser and watch:
- ๐ฑ๏ธ AI navigating websites and clicking buttons
- โจ๏ธ AI writing code in the terminal
- ๐๏ธ AI building and testing applications
- ๐ AI debugging by inspecting the screen
Perfect for demos, debugging, and building trust in AI agents.
What's inside each sandbox
Configuration
Requirements
Project structure
taw-computer/
โโโ mcp/
โ โโโ index.ts # MCP server โ stdio, 30+ tool handlers
โ โโโ browser.ts # Playwright CDP + Set-of-Mark engine
โโโ sandbox/
โ โโโ SandboxManager.ts # Abstract interface
โ โโโ DockerSandbox.ts # Docker implementation
โ โโโ FirecrackerSandbox.ts # Firecracker microVM (optional)
โ โโโ NetworkManager.ts # Network isolation
โ โโโ config.ts # Env-based config
โ โโโ index.ts # Auto-detect backend
โโโ images/
โ โโโ Dockerfile.taw # Ubuntu sandbox image
โโโ .github/
โ โโโ workflows/ci.yml # CI: typecheck + Docker build
โ โโโ ISSUE_TEMPLATE/ # Bug report + feature request
โโโ package.json
โโโ CONTRIBUTING.md
โโโ LICENSE (MIT)
Contributing
We'd love your help! See CONTRIBUTING.md.
Ideas for first contributions:
- ๐จ Record a demo GIF for this README
- ๐ Write a tutorial ("Build X with taw-computer")
- ๐ง Add a new MCP tool (audio? clipboard? multi-tab?)
- ๐ณ Build a slimmer Docker image
- ๐งช Add automated tests
- ๐ฆ Support Podman / containerd
Hosted version
Don't want to self-host? shipkit.cc โ managed taw-computer with:
- Chat UI (just type what you want)
- Auth & team collaboration
- One-click app sharing
- No Docker setup needed
FAQ
Those are closed-source, hosted-only products that generate code. taw-computer gives AI a real computer โ it can run servers, browse the web, install anything, and interact with any desktop app. It's also open source and self-hostable.
OpenInterpreter runs code on your local machine (risky). Open Hands uses its own LLM orchestration. taw-computer is just the computer โ no built-in LLM, no opinions about orchestration. Your existing AI client (Claude Code, Cursor, etc.) is the brain. taw-computer is a pure MCP server.
Each sandbox is an isolated Docker container with its own filesystem, network, and process space. Nothing inside can touch your host system. Containers have memory/CPU/PID limits. When you're done, destroy the VM.
Yes โ any AI client that supports MCP can connect. The server doesn't care which LLM is behind the client.
Yes. Anywhere Docker runs, taw-computer runs. The sandbox image supports both arm64 (Apple Silicon) and amd64 (Intel/AMD).
Star History
Related
- Model Context Protocol โ the protocol taw-computer speaks
- Claude Code โ AI coding CLI that supports MCP
- Playwright โ browser automation powering CDP tools
License
MIT โ do whatever you want with it.
