- name
SKILL.md
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add amplifthq/openmeld --skill debugging-ci-failures -a cursor -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add amplifthq/openmeld --skill debugging-ci-failures -a claude-code -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add amplifthq/openmeld --skill debugging-ci-failures -a codex -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add amplifthq/openmeld --skill debugging-ci-failures -a grok -ySend the copied message in a Grok Bot chat. The Bot fetches this SKILL.md from GitHub and adds it to the skills all your Bots share.
Install this skill: https://github.com/amplifthq/openmeld/tree/HEAD/.agents/skills/debugging-ci-failuresFrom the publisher
Before you propose a change to CI, check things already tried for the idea. It records what was measured, and why some good-sounding changes were reverted or rejected.
Find the first meaningful failure, classify it, reproduce the smallest useful case locally when appropriate, and report the result. Avoid public-visible or irreversible actions unless the user explicitly asks.
Always start with the hogli ci:insights digest. It aggregates across runs and
branches, which gh cannot do cheaply, and tells you whether a failure is
likely trunk-borne, gate-only, or isolated to a small set of branches. gh is
authoritative for one run's current state and attribution. Use the digest to
decide what to inspect; use gh to confirm whose failure it is and exactly
what failed in a given run.
This skill triages and classifies. Once a failure is confirmed flaky, hand off
to the fixing-flaky-tests skill, which owns local reproduction, root-cause
fixing, and N-run validation. For "who broke master" — the culprit commit and
the commit that fixed it — hand off to the investigating-ci-failures skill,
which owns the green/red boundary analysis. For aggregate pipeline health (is CI
getting slower, which workflow is the long pole, how long PRs take to merge),
read diagnosing-ci-and-merge-bottlenecks. Both are product skills under
products/engineering_analytics/skills/, not invocable here: read the
SKILL.md at that path.
GitHub Actions goes down often enough that it belongs before any log reading, and
a platform incident makes every other signal a symptom. It costs two page loads:
https://www.githubstatus.com/ for GitHub, https://status.depot.dev/ for the
runners. Check them whenever failures are broad — several workflows at once, a
burst of runs failing together, jobs dying before Checkout, or anything red
across unrelated PRs.
Report an outage as an outage, name the component, and stop recommending reruns.
Do not do any of these without explicit approval in the current conversation:
.github/workflows/ files (CI infra changes need human review).Read-only gh calls and read-only GitHub tools are fine. If you need to
change local Git state, make sure it is necessary for the task and does not
overwrite unrelated work.
hogli ci:insights reads PostHog's own engineering analytics — the cross-run
failure history a single run can't show. Consult it before any raw gh log
archaeology.
hogli ci:insights # digest for the current repo + branch
hogli ci:insights search "<error or test name>" # match a specific failure
hogli ci:insights view <ref> # one failure in full
hogli ci:insights view <ref> --logs # ...plus the failing log lines
search "<error>" to match it before reading logs, then
view <ref> on a row the digest or search printed. --logs prints the thinned
failing lines from that failure's latest run, which is usually enough to
classify without touching gh.Read each row's state as a triage ranking:
potentially_resolved is a hint, not a conclusion: confirm from run data before
reporting a failure as already fixed.
blocking_merge_queue proves a gate failure happened in the window, not that it
still blocks landings. Check the current queue run with gh before reporting it
as active. Likewise, confirm pr_only from the current run before assigning the
failure to a PR; it is also the fallback when job status is missing or stale.
Caveats to carry into whatever you report:
workflow_run webhook settles it, so
during a live incident confirm a specific run against gh.If nobody has signed in on this machine, hogli ci:insights exits 78. Treat
exit 78 as "no CI insights available" and fall back to the gh-based
inspection below — then tell the user they can run hogli posthog:login
once, which opens a browser and needs no API key. Do not run it yourself: it
waits on a consent screen you cannot see. Surface what you find per the Safety
rules — do not auto-apply a fix.
Determine the target in this order:
gh pr view --json number,headRefName,statusCheckRollup.A PR kicked from the merge queue is the exception: its own checks are the
wrong target. Trunk tests each queued PR on a trunk-merge/pr-<n>/<uuid>
branch holding master plus every PR queued ahead of it whose impacted targets
overlap its own. The lane script over-reports targets on purpose, so in
practice that is most of the queue. So:
trunk merge status <n> (/merging-prs step 4), not gh pr checks. Trunk publishes no check run here.
The branch is ephemeral; the run and its logs stay on GitHub, and the
warehouse keeps its jobs under that head_branch (query 8 in the
investigating-ci-failures references).blocking_merge_queue state.Inspect read-only:
gh pr checks <pr>
gh pr view <pr> --json statusCheckRollup
gh run view <run-id> --json jobs,conclusion,name,workflowName,url
gh run view <run-id> --log-failed
Use the full job log only when --log-failed lacks the failing command or
enough surrounding output:
gh run view <run-id> --log --job <job-id>
Given a run id, the engineering-analytics-run-failure-logs MCP tool returns
every failed job's error region with original line numbers, already thinned.
One call instead of a jobs listing plus a log download, and it works when the
job died before any test ran. It is bounded by Logs retention, so fall back to
gh for older runs. Given a PR number instead of a run id,
engineering-analytics-ci-failure-logs does the same across every run that PR
has pushed, so an earlier push's failure is still there.
Extract these before classifying:
.github/workflows/ci-backend.yml.backend-tests (4/10).Run pytest.When scanning logs, search for FAIL, Error, error:, assert,
Traceback, exit code, and ##[error]. Stop at the first failing step that
explains the run's conclusion. Keep excerpts under 40 lines.
For test-job failures, the trunk MCP server's investigate-ci-failure tool
is a shortcut past log scanning: give it the run URL
(https://github.com/PostHog/posthog/actions/runs/<run-id>) and it returns
structured failing-test details — names, error messages, stdout/stderr — from
the results CI uploaded to Trunk Flaky Tests, with quarantined known-flaky
tests already filtered out. It only covers what ran and uploaded: for jobs
that died before tests (build, setup, lint), use
engineering-analytics-run-failure-logs or gh run view --log.
Authenticate once via /mcp → trunk (browser OAuth); headless environments
instead add an Authorization: Bearer header with a TRUNK_API_TOKEN org
token to the server entry in .mcp.json. To dig into one test's flakiness
history, hand off to fixing-flaky-tests, which covers the search-test and
fix-flaky-test tools.
If multiple signals match, choose the most specific class. For example, prefer codegen drift over lint, migration over typecheck, and snapshot / visual over a generic Playwright test failure.
"Transient" is a claim about how often the job fails, so do not assert it from
one run. A job that dies before its tests run leaves no test-level evidence: no
FAILED line, so no fingerprint and no span, so it never appears as a row in
broken_tests or in the flaky-tests tool. It is still visible as a job
conclusion, which is what the digest's master-failures section groups by, and
engineering-analytics-run-failure-logs still returns its failing lines because
it reads by run, not by test. Start there, not from a test.
Unlike the span-derived test reads, this one can give you a real rate: the
warehouse records every job attempt, greens included, so the denominator is
honest. Query 7 in
products/engineering_analytics/skills/investigating-ci-failures/references/investigation-queries.md
is copy-ready; that skill also owns the wider investigation.
Read the result as:
/trunk merge yourself needs approval, per the Safety rules above.startup_failure before they
start and so leaves no log at all.Run only the narrowest command that exercises the failure. If the command shape
is unclear, read .agents/skills/hogli/SKILL.md and hogli <command> --help.
Do NOT run hogli test with no arguments. Do NOT run hogli nuke or
hogli dev:reset as a shortcut. Do NOT bypass hooks with --no-verify.
depot-ubuntu-latest or depot-ubuntu-latest-16.
Depot runs surface logs through the GitHub Actions UI / gh run view just
like standard GitHub-hosted runners, so read them there first.status.depot.dev covers the case
where Depot itself is the outage, and the depot-github-runners skill owns
runner troubleshooting beyond triage.Checkout completes (no app code ran), classify as
infra / runner. Do not propose code fixes.backend-tests (3/10) style). Reproduce from the specific failing test
path, not the shard index.Keep the response short. Include one likely-cause sentence and avoid deeper speculation.
Target: PR #<num> - run <run-id> (<workflow file>)
Failing job: <job name>
Failing step: <step name>
Command: <failing command>
Excerpt:
<up to 40 lines, trimmed around the failure>
Classification: <class from the table>
Shadow run: <yes | no>
Likely cause: <one sentence>
Local repro: <exact command, or "none">
Next action (needs your approval):
- <push fix | rerun job | update snapshot | none>
If the classification is infra / runner or a shadow run, say so and stop;
do not propose a code change. For infra / runner, include the base rate and
whether a retry is warranted.
>
---name: debugging-ci-failuresdescription: >---# .agents/skills/debugging-ci-failures/SKILL.mdFAQ
Download .agents/skills/debugging-ci-failures/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
The publisher keeps it at https://github.com/amplifthq/openmeld.
Claude, Cursor, Codex, and Grok Build can read a SKILL.md you add from the publisher's repository. Grok Bot installs it when you send the skill's GitHub folder link in a chat.
hogli lint:skills, hogli build:skills failure |
| skills build |
run the same hogli command locally |
SDK compat check, ci-survey-sdk-check, cross-version failure | SDK compatibility | check SDK version matrix for the affected package |