Alimenter votre agent
Installation de l'éditeur. Ce n'est pas un endpoint PlugMCP.
Host a copy with our keys and logs.
Pas encore de version packagée dans le registre. Suivez le README du dépôt.
URL distante ou commande sur votre machine ? Ce qui distingue les deux installations
Exécuter sur PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
LancerDe l'éditeur
A Rust-based benchmarking tool for measuring MCP (Model Context Protocol) server effectiveness in LLM-assisted development. Compare how different MCP servers impact token usage, cost, and task completion when using Claude Code.
Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
cargo install --path .
Or build from source:
cargo build --release
ANTHROPIC_API_KEY environment variable# Basic configuration
mcp-coder-bench init > mcp-coder-bench.yaml
# With Boarder MCP server example
mcp-coder-bench init --with-boarder > mcp-coder-bench.yaml
# Check config, workspace, and Docker connectivity
mcp-coder-bench validate
# Also verify container image exists
mcp-coder-bench validate --check-image
# Run with default configuration
mcp-coder-bench run
# Run with custom settings
mcp-coder-bench run -p 4 -n 5 # 4 parallel containers, 5 runs each
# Run a single scenario
mcp-coder-bench run --scenario baseline
# Dry-run to validate without executing
mcp-coder-bench run --dry-run
# Terminal table output
mcp-coder-bench analyze results/20260123/
# With statistical analysis
mcp-coder-bench analyze results/20260123/ --stats
# Export as Markdown with charts
mcp-coder-bench analyze results/20260123/ --format markdown --stats -o report.md
# Export as HTML with SVG visualizations
mcp-coder-bench analyze results/20260123/ --format html -o report.html
# Exclude outliers from analysis
mcp-coder-bench analyze results/20260123/ --exclude-outliers
# Basic comparison
mcp-coder-bench compare results/baseline results/with-mcp
# With significance testing
mcp-coder-bench compare results/baseline results/with-mcp --significance
# Multi-scenario comparison
mcp-coder-bench compare results/v1 results/v2 results/v3 --format markdown
Example configuration file:
name: "MCP Benchmark"
scenarios:
- name: "baseline"
description: "No MCP servers"
mcp_config: {}
- name: "with-boarder"
description: "With Boarder MCP server"
mcp_config:
mcpServers:
boarder:
command: "/usr/local/bin/boarder"
args: ["mcp"]
task:
prompt_file: "prompts/task.md"
workspace: "test-repo/"
timeout_seconds: 600
reset_strategy: "copy" # copy, git, or none
execution:
runs_per_scenario: 3
parallelism: 2
container_runtime: "auto"
container:
image: "mcp-coder-bench:latest" # Optional custom image
builder: "docker" # Optional: docker, podman, or auto
output:
directory: "results/"
formats: ["json", "markdown"]
git checkout and git cleanrunExecute benchmark scenarios.
mcp-coder-bench run [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
-p, --parallelism <N> Number of parallel containers [default: 1]
-n, --runs <N> Number of runs per scenario
-s, --scenario <NAME> Run only a specific scenario
-o, --output <DIR> Output directory for results
--runtime <RUNTIME> Container runtime (auto, docker, podman) [default: auto]
--rebuild Rebuild container image
--dry-run Validate without running
-v, --verbose Verbose output
analyzeAnalyze results from a benchmark run.
mcp-coder-bench analyze <RESULTS_DIR> [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--stats Include statistical analysis
--exclude-outliers Exclude outlier runs from analysis
--include-raw Include raw output in results
compareCompare benchmark result sets.
mcp-coder-bench compare <DIR>... [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--significance Include statistical significance testing
--confidence <LEVEL> Confidence level for tests [default: 0.95]
validateValidate configuration without running benchmarks.
mcp-coder-bench validate [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
--runtime <RUNTIME> Container runtime [default: auto]
--check-image Also verify container image exists
-v, --verbose Verbose output
initGenerate a sample configuration file.
mcp-coder-bench init [OPTIONS]
Options:
-o, --output <FILE> Output file (stdout if not specified)
--with-boarder Include example Boarder MCP configuration
╭──────────────┬──────┬────────────┬──────────┬──────────┬─────────╮
│ Scenario │ Runs │ Avg Tokens │ Avg Cost │ Avg Time │ Success │
├──────────────┼──────┼────────────┼──────────┼──────────┼─────────┤
│ baseline │ 5 │ 125.3K │ $1.45 │ 342.1s │ 100% │
│ with-boarder │ 5 │ 98.7K │ $1.12 │ 287.3s │ 100% │
╰──────────────┴──────┴────────────┴──────────┴──────────┴─────────╯
MCP Tool Usage:
mcp__boarder__cut_from_source - 2 calls
mcp__boarder__clear_buffer - 1 call
Logs are written to ~/.mcp-coder-bench/logs/ with rotation. Set RUST_LOG for console output:
RUST_LOG=debug mcp-coder-bench run
MIT License - see LICENSE for details.
Contributions welcome! Please ensure:
cargo test)cargo fmt)cargo clippy)Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
Outils
Pas encore de liste d'outils. Les serveurs distants en obtiennent une après le premier contrôle d'état.
FAQ
Copiez sur cette page la configuration stdio pour Cursor, Claude Desktop, Claude Code ou VS Code. Elle utilise l'installation de l'éditeur, pas un proxy PlugMCP.
Aucune variable d'environnement requise n'est indiquée sur cette fiche.
MCP Coder Bench s'exécute en local via stdio.
Dernière vérification : 2026-10-28. Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
