Potencia tu agente
Instalación del publicador. No es un endpoint de PlugMCP.
Host a copy with our keys and logs.
Aún no hay una versión empaquetada en el registro. Sigue el README del repositorio.
¿URL remota o un comando en tu máquina? En qué se diferencian las dos instalaciones
Ejecutar en PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
AbrirDel publicador
A Rust-based benchmarking tool for measuring MCP (Model Context Protocol) server effectiveness in LLM-assisted development. Compare how different MCP servers impact token usage, cost, and task completion when using Claude Code.
Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
cargo install --path .
Or build from source:
cargo build --release
ANTHROPIC_API_KEY environment variable# Basic configuration
mcp-coder-bench init > mcp-coder-bench.yaml
# With Boarder MCP server example
mcp-coder-bench init --with-boarder > mcp-coder-bench.yaml
# Check config, workspace, and Docker connectivity
mcp-coder-bench validate
# Also verify container image exists
mcp-coder-bench validate --check-image
# Run with default configuration
mcp-coder-bench run
# Run with custom settings
mcp-coder-bench run -p 4 -n 5 # 4 parallel containers, 5 runs each
# Run a single scenario
mcp-coder-bench run --scenario baseline
# Dry-run to validate without executing
mcp-coder-bench run --dry-run
# Terminal table output
mcp-coder-bench analyze results/20260123/
# With statistical analysis
mcp-coder-bench analyze results/20260123/ --stats
# Export as Markdown with charts
mcp-coder-bench analyze results/20260123/ --format markdown --stats -o report.md
# Export as HTML with SVG visualizations
mcp-coder-bench analyze results/20260123/ --format html -o report.html
# Exclude outliers from analysis
mcp-coder-bench analyze results/20260123/ --exclude-outliers
# Basic comparison
mcp-coder-bench compare results/baseline results/with-mcp
# With significance testing
mcp-coder-bench compare results/baseline results/with-mcp --significance
# Multi-scenario comparison
mcp-coder-bench compare results/v1 results/v2 results/v3 --format markdown
Example configuration file:
name: "MCP Benchmark"
scenarios:
- name: "baseline"
description: "No MCP servers"
mcp_config: {}
- name: "with-boarder"
description: "With Boarder MCP server"
mcp_config:
mcpServers:
boarder:
command: "/usr/local/bin/boarder"
args: ["mcp"]
task:
prompt_file: "prompts/task.md"
workspace: "test-repo/"
timeout_seconds: 600
reset_strategy: "copy" # copy, git, or none
execution:
runs_per_scenario: 3
parallelism: 2
container_runtime: "auto"
container:
image: "mcp-coder-bench:latest" # Optional custom image
builder: "docker" # Optional: docker, podman, or auto
output:
directory: "results/"
formats: ["json", "markdown"]
git checkout and git cleanrunExecute benchmark scenarios.
mcp-coder-bench run [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
-p, --parallelism <N> Number of parallel containers [default: 1]
-n, --runs <N> Number of runs per scenario
-s, --scenario <NAME> Run only a specific scenario
-o, --output <DIR> Output directory for results
--runtime <RUNTIME> Container runtime (auto, docker, podman) [default: auto]
--rebuild Rebuild container image
--dry-run Validate without running
-v, --verbose Verbose output
analyzeAnalyze results from a benchmark run.
mcp-coder-bench analyze <RESULTS_DIR> [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--stats Include statistical analysis
--exclude-outliers Exclude outlier runs from analysis
--include-raw Include raw output in results
compareCompare benchmark result sets.
mcp-coder-bench compare <DIR>... [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--significance Include statistical significance testing
--confidence <LEVEL> Confidence level for tests [default: 0.95]
validateValidate configuration without running benchmarks.
mcp-coder-bench validate [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
--runtime <RUNTIME> Container runtime [default: auto]
--check-image Also verify container image exists
-v, --verbose Verbose output
initGenerate a sample configuration file.
mcp-coder-bench init [OPTIONS]
Options:
-o, --output <FILE> Output file (stdout if not specified)
--with-boarder Include example Boarder MCP configuration
╭──────────────┬──────┬────────────┬──────────┬──────────┬─────────╮
│ Scenario │ Runs │ Avg Tokens │ Avg Cost │ Avg Time │ Success │
├──────────────┼──────┼────────────┼──────────┼──────────┼─────────┤
│ baseline │ 5 │ 125.3K │ $1.45 │ 342.1s │ 100% │
│ with-boarder │ 5 │ 98.7K │ $1.12 │ 287.3s │ 100% │
╰──────────────┴──────┴────────────┴──────────┴──────────┴─────────╯
MCP Tool Usage:
mcp__boarder__cut_from_source - 2 calls
mcp__boarder__clear_buffer - 1 call
Logs are written to ~/.mcp-coder-bench/logs/ with rotation. Set RUST_LOG for console output:
RUST_LOG=debug mcp-coder-bench run
MIT License - see LICENSE for details.
Contributions welcome! Please ensure:
cargo test)cargo fmt)cargo clippy)Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
Herramientas
Aún no hay lista de herramientas. Los servidores remotos la reciben tras la primera comprobación de estado.
Preguntas frecuentes
Copia en esta página la configuración stdio para Cursor, Claude Desktop, Claude Code o VS Code. Usa la instalación del publicador, no un proxy de PlugMCP.
Este registro no indica ninguna variable de entorno obligatoria.
MCP Coder Bench se ejecuta localmente por stdio.
Última comprobación: 2026-10-28. Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
