fiji-mcp
https://github.com/user-attachments/assets/2933af17-6cb6-4a4c-a7e8-6c1391c49db7
Prior art & related projects — several other projects bridge bioimage tools with LLMs via MCP. This project was developed independently but shares the same problem space:
- fiji/fiji-llm project - Chat with AI assistants directly in Fiji to get help with image analysis, scripting, macros, and general questions.
- fiji_mcp (NicoKiaru) — Fiji MCP via PyImageJ (in-process, Groovy)
- fiji-mcp-bridge (kusumotok) — Fiji MCP via TCP sockets, curated tool wrappers
- napari-mcp (royerlab) — napari MCP server, PyPI published
- bioimage-mcp (cqian89) — artifact-based I/O, isolated conda envs
- bioimage-mcp-server (Ichoran) — Java MCP for Bio-Formats
- BioImage-Agent (LLNL) — napari plugin over MCP via sockets
- cellpose_mcp — Cellpose segmentation MCP
An MCP server that gives LLM agents (Claude Code, Claude Desktop, Cursor, etc.) a live scripting interface into a running Fiji instance via WebSocket.
LLM Agent --(MCP/stdio)--> Python MCP Server --(WebSocket)--> Fiji + Java Plugin
What it does
- Run macros and scripts in Fiji from your LLM conversation (
run_ij_macro,run_script,run_command) - Inspect images — list open images, get metadata, save to disk
- Visual feedback via thumbnails —
get_thumbnailreturns a display-ready PNG with the current LUT and overlays baked in, so the LLM can see what it's working with - Observe user actions — the plugin captures what the biologist does in Fiji's GUI (commands, image opens/closes, ROI edits) and streams it to the LLM via an event log
- Export workflows — convert a recorded action sequence to a runnable ImageJ macro
- Search commands — find Fiji menu commands by name
Thumbnails and visual models
This project is designed for use with vision-capable LLMs (Claude, GPT-4o, Gemini, etc.). The get_thumbnail tool is the primary way the agent sees image data — it returns a scaled PNG with the current LUT, brightness/contrast, and overlays already applied, exactly as the image appears in Fiji.
Agents should call get_thumbnail liberally: after opening an image, after each processing step, and before reporting results. This gives the model visual ground truth to catch mistakes (wrong channel, bad threshold, off-target ROI) that text-only metadata would miss. The thumbnail is file-path-based — no base64 in the protocol — so it works efficiently even for large images.
Available tools
Requirements
Installation
1. Install fiji-mcp
git clone https://github.com/SteffenPL/fiji-mcp.git
cd fiji-mcp
uv sync
2. Install the bridge plugin into Fiji
# Auto-discovers Fiji in standard locations:
uv run fiji-mcp install
# Or specify your Fiji path explicitly:
uv run fiji-mcp install --fiji-home /path/to/Fiji.app
This copies the bridge plugin JAR into your Fiji's plugins/ directory, checks the bundled Java version, and saves the Fiji path so the MCP server can find it at runtime.
3. Configure your MCP client
Add fiji-mcp to your MCP client.
Claude Code
Project scope (run from your project root):
claude mcp add --scope project fiji-mcp -- \
uv run --directory /absolute/path/to/fiji-mcp fiji-mcp
Global scope:
claude mcp add --scope user fiji-mcp -- \
uv run --directory /absolute/path/to/fiji-mcp fiji-mcp
Codex CLI / Codex app
In the Codex app, use the Show Terminal button and run the same commands below.
Global scope:
codex mcp add fiji-mcp -- \
uv run --directory /absolute/path/to/fiji-mcp fiji-mcp
For project scope, use a local .codex/config.toml file as shown below.
Optional: config files
Claude config
{
"mcpServers": {
"fiji-mcp": {
"command": "uv",
"args": ["run", "--directory", "/absolute/path/to/fiji-mcp", "fiji-mcp"]
}
}
}
Use this JSON either in a project .mcp.json for Claude Code, or in Claude Desktop's config file at ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows).
Codex project or global config
[mcp_servers.fiji-mcp]
command = "uv"
args = ["run", "--directory", "/absolute/path/to/fiji-mcp", "fiji-mcp"]
Use this block in .codex/config.toml for a project-only install, or in ~/.codex/config.toml for a global install.
To override the saved Fiji path at runtime, pass --fiji-home:
uv run --directory /absolute/path/to/fiji-mcp fiji-mcp --fiji-home /other/Fiji.app
Usage
The first tool call auto-launches Fiji if it's not already running. You can also start Fiji manually and go to Plugins > fiji-mcp > Start Bridge.
Example conversation:
You: Open the sample image "blobs.gif" and apply a Gaussian blur with sigma 2
Claude: (uses
run_ij_macrowith codeopen("http://imagej.net/images/blobs.gif"); run("Gaussian Blur...", "sigma=2");)
Configuration
Architecture
Two processes connected by WebSocket:
- fiji-mcp (Python) — MCP server using fastmcp 3.x with stdio transport. Connects to Fiji as a WebSocket client. Buffers events in a 10,000-entry ring buffer.
- fiji-mcp-bridge (Java) — Fiji plugin. Runs a WebSocket server inside Fiji. Executes macros/scripts, captures user actions via ImageJ listeners, and pushes events to the MCP server.
Image data is file-based only (Fiji saves to temp directory, MCP server returns file paths). No base64 encoding in the protocol.
Event capture
The plugin captures user and MCP-triggered actions across 7 categories:
Every event is tagged with "source": "user" or "source": "mcp" so the LLM can distinguish what the biologist did from what it triggered itself.
Development
# Install dev dependencies
uv sync --dev
# Run Python tests (no Fiji required)
uv run pytest -v
# Run smoke test (requires Fiji with bridge started)
uv run python tests/smoke_test.py
# Build the Java plugin from source (requires Maven + JDK 8+)
cd fiji-plugin && mvn package -q
# Copy to package data:
cp target/fiji-mcp-bridge-0.1.0.jar ../src/fiji_mcp/data/