Power your agent
Cài đặt từ nhà phát hành. Không phải endpoint của PlugMCP.
Host a copy with our keys and logs.
Chưa có bản phát hành đóng gói nào trong registry. Làm theo README của kho mã.
URL từ xa hay một lệnh trên máy của bạn? Hai cách cài đặt khác nhau thế nào
Chạy trên PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
MởTừ nhà phát hành
A Rust-based benchmarking tool for measuring MCP (Model Context Protocol) server effectiveness in LLM-assisted development. Compare how different MCP servers impact token usage, cost, and task completion when using Claude Code.
Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
cargo install --path .
Or build from source:
cargo build --release
ANTHROPIC_API_KEY environment variable# Basic configuration
mcp-coder-bench init > mcp-coder-bench.yaml
# With Boarder MCP server example
mcp-coder-bench init --with-boarder > mcp-coder-bench.yaml
# Check config, workspace, and Docker connectivity
mcp-coder-bench validate
# Also verify container image exists
mcp-coder-bench validate --check-image
# Run with default configuration
mcp-coder-bench run
# Run with custom settings
mcp-coder-bench run -p 4 -n 5 # 4 parallel containers, 5 runs each
# Run a single scenario
mcp-coder-bench run --scenario baseline
# Dry-run to validate without executing
mcp-coder-bench run --dry-run
# Terminal table output
mcp-coder-bench analyze results/20260123/
# With statistical analysis
mcp-coder-bench analyze results/20260123/ --stats
# Export as Markdown with charts
mcp-coder-bench analyze results/20260123/ --format markdown --stats -o report.md
# Export as HTML with SVG visualizations
mcp-coder-bench analyze results/20260123/ --format html -o report.html
# Exclude outliers from analysis
mcp-coder-bench analyze results/20260123/ --exclude-outliers
# Basic comparison
mcp-coder-bench compare results/baseline results/with-mcp
# With significance testing
mcp-coder-bench compare results/baseline results/with-mcp --significance
# Multi-scenario comparison
mcp-coder-bench compare results/v1 results/v2 results/v3 --format markdown
Example configuration file:
name: "MCP Benchmark"
scenarios:
- name: "baseline"
description: "No MCP servers"
mcp_config: {}
- name: "with-boarder"
description: "With Boarder MCP server"
mcp_config:
mcpServers:
boarder:
command: "/usr/local/bin/boarder"
args: ["mcp"]
task:
prompt_file: "prompts/task.md"
workspace: "test-repo/"
timeout_seconds: 600
reset_strategy: "copy" # copy, git, or none
execution:
runs_per_scenario: 3
parallelism: 2
container_runtime: "auto"
container:
image: "mcp-coder-bench:latest" # Optional custom image
builder: "docker" # Optional: docker, podman, or auto
output:
directory: "results/"
formats: ["json", "markdown"]
git checkout and git cleanrunExecute benchmark scenarios.
mcp-coder-bench run [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
-p, --parallelism <N> Number of parallel containers [default: 1]
-n, --runs <N> Number of runs per scenario
-s, --scenario <NAME> Run only a specific scenario
-o, --output <DIR> Output directory for results
--runtime <RUNTIME> Container runtime (auto, docker, podman) [default: auto]
--rebuild Rebuild container image
--dry-run Validate without running
-v, --verbose Verbose output
analyzeAnalyze results from a benchmark run.
mcp-coder-bench analyze <RESULTS_DIR> [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--stats Include statistical analysis
--exclude-outliers Exclude outlier runs from analysis
--include-raw Include raw output in results
compareCompare benchmark result sets.
mcp-coder-bench compare <DIR>... [OPTIONS]
Options:
-f, --format <FORMAT> Output format (json, csv, markdown, html, table) [default: table]
-o, --output <FILE> Output file (stdout if not specified)
--significance Include statistical significance testing
--confidence <LEVEL> Confidence level for tests [default: 0.95]
validateValidate configuration without running benchmarks.
mcp-coder-bench validate [OPTIONS]
Options:
-c, --config <FILE> Configuration file [default: mcp-coder-bench.yaml]
--runtime <RUNTIME> Container runtime [default: auto]
--check-image Also verify container image exists
-v, --verbose Verbose output
initGenerate a sample configuration file.
mcp-coder-bench init [OPTIONS]
Options:
-o, --output <FILE> Output file (stdout if not specified)
--with-boarder Include example Boarder MCP configuration
╭──────────────┬──────┬────────────┬──────────┬──────────┬─────────╮
│ Scenario │ Runs │ Avg Tokens │ Avg Cost │ Avg Time │ Success │
├──────────────┼──────┼────────────┼──────────┼──────────┼─────────┤
│ baseline │ 5 │ 125.3K │ $1.45 │ 342.1s │ 100% │
│ with-boarder │ 5 │ 98.7K │ $1.12 │ 287.3s │ 100% │
╰──────────────┴──────┴────────────┴──────────┴──────────┴─────────╯
MCP Tool Usage:
mcp__boarder__cut_from_source - 2 calls
mcp__boarder__clear_buffer - 1 call
Logs are written to ~/.mcp-coder-bench/logs/ with rotation. Set RUST_LOG for console output:
RUST_LOG=debug mcp-coder-bench run
MIT License - see LICENSE for details.
Contributions welcome! Please ensure:
cargo test)cargo fmt)cargo clippy)Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
Công cụ
Chưa có danh sách công cụ. Máy chủ từ xa sẽ có danh sách sau lần kiểm tra tình trạng đầu tiên.
Câu hỏi thường gặp
Sao chép cấu hình stdio cho Cursor, Claude Desktop, Claude Code hoặc VS Code trên trang này. Cấu hình dùng bản cài đặt của nhà phát hành, không phải proxy của PlugMCP.
Bản ghi này không liệt kê biến môi trường bắt buộc nào.
MCP Coder Bench chạy cục bộ qua stdio.
Kiểm tra lần cuối: 2026-10-28. Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
