Power your agent
Instal dari penerbit. Bukan endpoint PlugMCP.
URL remote atau perintah di mesin Anda? Perbedaan kedua cara instal
Jalankan di PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
BukaDari penerbit
A Rust-based benchmarking tool for measuring MCP (Model Context Protocol) server effectiveness in LLM-assisted development. Compare how different MCP servers impact token usage, cost, and task completion when using Claude Code.
Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
cargo install --path .
Or build from source:
cargo build --release
ANTHROPIC_API_KEY environment variable# Basic configuration
mcp-coder-bench init > mcp-coder-bench.yaml
# With Boarder MCP server example
mcp-coder-bench init --with-boarder > mcp-coder-bench.yaml
# Check config, workspace, and Docker connectivity
mcp-coder-bench validate
# Also verify container image exists
mcp-coder-bench validate --check-image
# Run with default configuration
mcp-coder-bench run
# Run with custom settings
mcp-coder-bench run -p 4 -n 5 # 4 parallel containers, 5 runs each
# Run a single scenario
mcp-coder-bench run --scenario baseline
# Dry-run to validate without executing
mcp-coder-bench run --dry-run
# Terminal table output
mcp-coder-bench analyze results/20260123/
# With statistical analysis
mcp-coder-bench analyze results/20260123/ --stats
# Export as Markdown with charts
mcp-coder-bench analyze results/20260123/ --format markdown --stats -o report.md
# Export as HTML with SVG visualizations
mcp-coder-bench analyze results/20260123/ --format html -o report.html
# Exclude outliers from analysis
mcp-coder-bench analyze results/20260123/ --exclude-outliers
# Basic comparison
mcp-coder-bench compare results/baseline results/with-mcp
# With significance testing
mcp-coder-bench compare results/baseline results/with-mcp --significance
# Multi-scenario comparison
mcp-coder-bench compare results/v1 results/v2 results/v3 --format markdown
Example configuration file:
name: "MCP Benchmark"
scenarios:
- name: "baseline"
description: "No MCP servers"
mcp_config: {}
- name: "with-boarder"
description: "With Boarder MCP server"
mcp_config:
mcpServers:
boarder:
command: "/usr/local/bin/boarder"
args: ["mcp"]
task:
prompt_file: "prompts/task.md"
workspace: "test-repo/"
timeout_seconds: 600
reset_strategy: "copy" # copy, git, or none
execution:
runs_per_scenario: 3
parallelism: 2
container_runtime: "auto"
container:
image: "mcp-coder-bench:latest" # Optional custom image
builder: "docker" # Optional: docker, podman, or auto
output:
directory: "results/"
formats: ["json", "markdown"]
git checkout and git cleanrunExecute benchmark scenarios.
mcp-coder-bench run [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
-p, --parallelism <N> Number of parallel containers [default: 1]
-n, --runs <N> Number of runs per scenario
-s, --scenario <NAME> Run only a specific scenario
-o, --output <DIR> Output directory for results
--runtime <RUNTIME> Container runtime (auto, docker, podman) [default: auto]
--rebuild Rebuild container image
--dry-run Validate without running
-v, --verbose Verbose output
analyzeAnalyze results from a benchmark run.
mcp-coder-bench analyze <RESULTS_DIR> [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--stats Include statistical analysis
--exclude-outliers Exclude outlier runs from analysis
--include-raw Include raw output in results
compareCompare benchmark result sets.
mcp-coder-bench compare <DIR>... [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--significance Include statistical significance testing
--confidence <LEVEL> Confidence level for tests [default: 0.95]
validateValidate configuration without running benchmarks.
mcp-coder-bench validate [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
--runtime <RUNTIME> Container runtime [default: auto]
--check-image Also verify container image exists
-v, --verbose Verbose output
initGenerate a sample configuration file.
mcp-coder-bench init [OPTIONS]
Options:
-o, --output <FILE> Output file (stdout if not specified)
--with-boarder Include example Boarder MCP configuration
╭──────────────┬──────┬────────────┬──────────┬──────────┬─────────╮
│ Scenario │ Runs │ Avg Tokens │ Avg Cost │ Avg Time │ Success │
├──────────────┼──────┼────────────┼──────────┼──────────┼─────────┤
│ baseline │ 5 │ 125.3K │ $1.45 │ 342.1s │ 100% │
│ with-boarder │ 5 │ 98.7K │ $1.12 │ 287.3s │ 100% │
╰──────────────┴──────┴────────────┴──────────┴──────────┴─────────╯
MCP Tool Usage:
mcp__boarder__cut_from_source - 2 calls
mcp__boarder__clear_buffer - 1 call
Logs are written to ~/.mcp-coder-bench/logs/ with rotation. Set RUST_LOG for console output:
RUST_LOG=debug mcp-coder-bench run
MIT License - see LICENSE for details.
Contributions welcome! Please ensure:
cargo test)cargo fmt)cargo clippy)Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
Tool
Belum ada daftar tool. Server remote mendapatkannya setelah pemeriksaan kesehatan pertama.
FAQ
Salin konfigurasi stdio untuk Cursor, Claude Desktop, Claude Code, atau VS Code di halaman ini. Konfigurasi ini memakai instalasi dari penerbit, bukan proxy PlugMCP.
Tidak ada variabel lingkungan wajib yang tercantum di catatan ini.
MCP Coder Bench berjalan secara lokal melalui stdio.
Terakhir diperiksa 2026-10-28. Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
