📊 flakiness-knowledge-graph-mcp
A Playwright custom reporter + MCP server that builds a local flakiness knowledge graph from your test run history. Ask your AI agent which tests are unreliable, on which browser, and whether they're getting worse.
接入你的智能体
發布者自己的指令。不是 PlugMCP 端點。
用我們的金鑰與日誌託管一份副本。
npx -y [email protected]Claude Code 沒有本機指令的安裝連結。複製下方指令並在終端機中執行。
claude mcp add io.github.vola-trebla-flakiness-knowledge-graph-mcp -- npx -y flakiness-knowledge-graph-mcp@0.2.2在終端機中執行。
{ "mcpServers": { "io.github.vola-trebla-flakiness-knowledge-graph-mcp": { "command": "npx", "args": [ "-y", "[email protected]" ] } }}claude_desktop_config.json
遠端 URL 還是本機指令? 兩種安裝方式的差異
來自發布者
A Playwright custom reporter + MCP server that builds a local flakiness knowledge graph from your test run history. Ask your AI agent which tests are unreliable, on which browser, and whether they're getting worse.
MCP server + Playwright reporter that builds a flakiness knowledge graph from test run history
A single Playwright trace tells you what failed right now. It doesn't tell you whether this test has been silently flaking for two weeks, or only fails on Firefox in CI, or is getting slower with every release.
This tool fixes that by accumulating run history into a SQLite database and exposing it to AI agents via MCP.
npm install -g flakiness-knowledge-graph-mcp
Or build from source:
git clone https://github.com/vola-trebla/flakiness-knowledge-graph-mcp.git
cd flakiness-knowledge-graph-mcp
npm install && npm run build
// playwright.config.ts
export default defineConfig({
reporter: [["html"], ["flakiness-knowledge-graph-mcp/reporter", { dbPath: "./flakiness.db" }]],
});
Run your tests normally — the reporter writes every result to flakiness.db automatically.
.cursor/mcp.json or .vscode/mcp.json){
"mcpServers": {
"flakiness-knowledge-graph": {
"command": "flakiness-knowledge-graph-mcp"
}
}
}
claude mcp add flakiness-knowledge-graph flakiness-knowledge-graph-mcp
No Playwright project yet? Generate 30 days of realistic sample data:
npx flakiness-graph-seed ./demo.db
Then point your AI agent at ./demo.db to explore all 8 tools.
I've been running my Playwright suite for two weeks. The DB is at /my-project/flakiness.db.
1. get_flaky_tests — which tests are most unreliable? Show last 7 days only.
2. get_test_history for the top flaky test — is it getting worse?
3. get_flakiness_trend for the same test over 14 days — plot the daily rate.
4. get_failure_patterns — does it only fail on a specific browser or OS?
5. cluster_semantic_error_trees — are multiple tests failing with semantically identical errors?
6. correlate_git_commit_flakiness — which commit introduced the flakiness?
7. get_slow_tests — which tests should I optimize for CI speed?
get_error_groups clusters by raw string prefix — if the error contains a UUID or element ID it creates separate groups for what is really one root cause. cluster_semantic_error_trees strips dynamic values first:
{
"total_clusters": 2,
"clusters": [
{
"cluster_id": "cluster-1",
"canonical_message": "TimeoutError: locator.click: Timeout 30000ms exceeded\n waiting for locator('#submit-btn')",
"normalized_message": "TimeoutError: locator.click: Timeout <num>ms exceeded waiting for locator",
"error_taxonomy": "TimeoutError",
"instance_count": 14,
"affected_tests": 3,
"sample_test_ids": ["checkout > submit order", "cart > add item", "checkout > apply coupon"]
},
{
"cluster_id": "cluster-2",
"canonical_message": "Error: 2 requests to https://api.example.com/orders/8f3a1c were made. Expected 1",
"normalized_message": "Error: <num> requests to <url> were made. Expected <num>",
"error_taxonomy": "AssertionError",
"instance_count": 6,
"affected_tests": 1,
"sample_test_ids": ["api-mock > intercept order"]
}
]
}
correlate_git_commit_flakiness uses a state machine — it looks for runs where a test was stable for ≥3 consecutive passes, then failed. The transition record includes the SHA from the CI environment:
{
"total_transitions": 1,
"transitions": [
{
"test_id": "auth > login > should redirect after login",
"title": "should redirect after login",
"transition_type": "stable_to_flaky",
"git_commit_sha": "a3f8c1d9e2b54f6a",
"git_branch": "main",
"git_author": "dev-handle",
"transition_date": "2025-04-14"
}
]
}
The reporter reads GITHUB_SHA / CI_COMMIT_SHA / CIRCLE_SHA1 / GIT_COMMIT automatically — no reporter config changes needed beyond upgrading to v0.2.0.
These two MCP servers are designed to complement each other:
Combined, an AI agent can diagnose whether a CI failure is a known flaky test or a new regression — without you opening a single file.
flakiness-knowledge-graph-mcp uses an in-process write queue to ensure that parallel Playwright workers within a single Node process do not corrupt the database.
However, if you run tests across multiple independent processes (e.g., parallel CI shards or separate machine runners) writing to the same shared network file:
flakiness-shard-1.db, flakiness-shard-2.db).For local development or single-machine CI runs, the default configuration is safe.
playwright.config.ts
└── FlakinessReporter → flakiness.db (SQLite via sql.js)
flakiness.db
└── test_runs table
id, test_id, title, suite, file,
status, duration_ms, browser, os,
timestamp, error, retry,
git_commit_sha, git_branch, git_author ← added in v0.2.0
MCP server
└── reads flakiness.db on demand (in-process handle reuse)
sql.js is used instead of better-sqlite3 — pure JavaScript SQLite compiled to WebAssembly, no native compilation needed. The git columns are added via ALTER TABLE migration on first use — existing databases upgrade automatically.
npm run build # compile TypeScript → dist/
npm run lint # ESLint
npm run format # Prettier --write
npm run format:check # Prettier check (used in CI)
npm run seed # populate flakiness.db with 30 days of demo data
MIT
MCP server + Playwright reporter that builds a flakiness knowledge graph from test run history
{ "mcpServers": { "io.github.vola-trebla-flakiness-knowledge-graph-mcp": { "command": "npx", "args": [ "-y", "[email protected]" ] } }}工具
尚無工具清單。遠端伺服器會在首次健康檢查後取得工具清單。
常見問題
在本頁複製適用於 Cursor、Claude Desktop、Claude Code 或 VS Code 的 stdio 設定。它使用發布者的安裝方式,而不是 PlugMCP 代理伺服器。
這筆記錄未列出必要的環境變數。
Flakiness Knowledge Graph 透過 stdio 在本機執行。
上次檢查時間:2026-11-10。 MCP server + Playwright reporter that builds a flakiness knowledge graph from test run history