@ia Qa/self Healing

ia-qa-heal — self-healing selectors for E2E tests

Your E2E tests break because a selector moved, not because the app is wrong. This finds the element again and rewrites the test.

The verdict is deterministic — no model decides whether your build passes. When the repair is not provable, the tool refuses to guess and tells you why, rather than quietly rewriting a test into a green lie.

It runs entirely on your machine. It brings its own headless Chromium, so it sees localhost, staging, or an app behind a VPN, and nothing ever leaves the machine.

Framework-agnostic — Playwright, Cypress, Selenium; JS/TS, Python, Java. Ships a CLI, a local MCP server for AI agents, and an accessibility-tree page mapper.

New here? The step-by-step tutorial walks the whole thing with no jargon (same text as TUTORIAL.md, shipped in this package). This README is the short reference.

About this repository — it is the issue tracker and documentation for @ia-qa/self-healing. The package is distributed on npm; its source is not published here. Bug reports, feature requests and questions are welcome in Issues.

See it work

ia-qa-heal run runs your suite, watches it break, diffs the live app against your baseline, rewrites the dead selectors, then re-runs the suite to verify and exits on that verdict. Nothing is committed. By default a selector is replaced by the element's new selector, in the style your suite already uses — CSS stays CSS. The demo adds --locators, which rewrites to getByRole(…, { exact: true }) where that is provably safe: a locator immune to the next layout change.

▶ The full 49-second version also covers what deterministic healing refuses to guess, the MCP server answering an agent, and the HTML report --report writes for a PR.

Install

npm i -D @ia-qa/self-healing

Four commands to your first verdict — no baseline needed, nothing is rewritten:

npx ia-qa-heal init      # interactive: writes .ia-qa/config.json
npx ia-qa-heal ingest    # inventory the selectors and names your tests use
npx ia-qa-heal map       # capture the app, and bind your selectors to it
npx ia-qa-heal audit     # does my suite still name things that exist?

audit is read-only and advisory (exit 0 unless --strict) — it answers on day one, before you have any history to compare against.

Then one command, for good:

npx ia-qa-heal baseline  # lock today's UI as the reference — once
# … someone changes the UI …
npx ia-qa-heal run       # runs your suite → diffs → fixes → re-runs to verify

run is the whole loop in one verb: it runs your suite, watches it break, diffs the live app against the baseline, rewrites the dead selectors, then re-runs the suite to verify and exits on that verdict. Nothing is committed.

npx ia-qa-heal map
npx ia-qa-heal diff        # PASS / FIX / BLOCK — this is the CI gate
npx ia-qa-heal fix --dry-run
npx ia-qa-heal fix

Use this when your suite is slow (run runs it twice), or when CI already captures with IAQA_CAPTURE=1 — then diff alone costs nothing extra.

ingest comes first, and it is not optional if your tests locate by CSS: it is what lets map bind the strings your tests write to real elements. Re-run it when your test files change — map says so when they have. Commit .ia-qa/ (except session.json): the baseline is what CI compares against.

Three binaries ship together: ia-qa-heal (the loop), ia-qa-heal-mcp (stdio MCP server for agents), ia-qa-heal-ai (optional BYOK suggestions — never a gate).

Behind a login

Most apps worth mapping are behind one, and only some logins are a form.

npx ia-qa-heal login     # opens a real browser; you log in however the app asks
npx ia-qa-heal map       # reuses that session

It models nothing about your login, which is why SSO, MFA, a consent screen and magic links all work — you perform them. What is saved is the browser session (.ia-qa/session.json): live cookies, so treat it like a password. It stays on your machine, login writes .ia-qa/.gitignore so a commit cannot carry it, and when it expires map says so instead of quietly mapping the login page under your dashboard's name.

Already have a session? An authenticated Playwright suite usually writes a storageState in globalSetup — point at it instead of logging in twice:

npx ia-qa-heal map --session playwright/.auth/user.json

Precedence: --session → IAQA_SESSION → "session" in config → what login wrote. A session you named and that cannot be read stops the run; it never falls through to "no session", which would capture the login page under your pages' names.

login needs a terminal and refuses under CI. There, use auth (a form fill, secrets from env/SSM — see Configuration) or capture-during-run, where your own suite logs itself in.

Which verb answers which question

ia-qa-heal <verb> --help for flags. On a slow suite, note that run runs it twice (once to capture, once to verify the fix): run --no-verify keeps only the first, and IAQA_CAPTURE=1 on the run you already do in CI plus ia-qa-heal diff costs nothing extra.

The verdict

🔧  FIX   2 ok · 1 renamed · 3 healable · 0 ambiguous · 0 lost

• PASS — nothing moved.
• FIX — every break has a deterministic old→new rewrite. fix applies it.
• BLOCK — a human decides. Nothing is rewritten.

What fix refuses is the point of the tool:

--locators rewrites to getByRole(role, { name }) instead of a fresher CSS path, where that is provably safe — a selector that survives the next layout change. Without it, a CSS selector is replaced by a CSS selector: the tool does not impose a locator style on your suite.

At runtime, if you want it — aiClick / aiFill

import { aiClick } from '@ia-qa/self-healing';
await aiClick(page, 'button#login');

If the normal action times out, these read the page contract, re-scan the live page, retry on the element that semantically matches, and log a loud warning that the test needs updating. Useful to keep a suite green while you triage — but it is a stopgap, not the loop: healing in memory hides the drift instead of putting it in a pull request. The verbs above edit your files so a human reviews the change.

Page Objects written in CSS

Your Page Object says .btn-primary. The contract says #pay-now. Both name the same button, and until they are bound every drift row reads not referenced by your tests — technically true, useless in practice.

So map binds them (with usage.json present — run ingest first) and records the result in .ia-qa/mapping/_resolved/. What that buys, on a diff where the element contract did not move at all:

🔧  FIX
   2 ok · 0 renamed · 0 healable · 0 lost      ← the contract is spotless

   🔗 1 selector your tests write no longer reaches what it used to:
      🔧 .btn-primary  →  #pay-now
         the element is still there, only its address moved.
         tests/checkout.spec.js:1, tests/checkout.spec.js:4

Somebody renamed a CSS class. The element diff is clean, the suite is broken, and fix rewrites the literal in your files. Four outcomes, one repaired automatically:

Your app may already carry the answer. The contract holds interactive elements only, so a selector on a status div, a cell or an alert is not in it. Usually that stays quiet — such an element has no identity independent of what drifts, so nothing could be repaired anyway. But when it does carry one (data-testid, id, aria-label), the repair is exact:

🔧 .infoBlockContainer > .alert  →  [data-testid="cart-alert"]

That is the common shape of a legacy suite: the front end added test ids years after the Page Objects were written and nobody propagated them. It cannot be noisy — an element with a test id has a stable identity by construction, and an anchor matching two elements is refused.

How much of your suite is in reach? map says so at the end, over the union of the pages it just mapped — the number nobody can add up by hand, because the same string appears on several pages:

🔗 Your 181 inventoried selectors, against the pages just mapped:
   34 repairable   reach a contracted element, or an identity this tool can rewrite towards
   52 working      reach something real with no stable identity — fine today, nothing to repair towards
   95 not seen     on any page mapped this run — which is about your coverage, not your tests

Read the composition, not the ratio. working selectors are fine. And not seen is a statement about how many pages you map: a selector on a page absent from config.pages, or in a state no capture reached, looks exactly like one that is gone. ia-qa-heal discover lists what you are missing.

No usage.json ⇒ no bindings ⇒ nothing changes. Same rule as name drift: no inventory, no escalation.

Capture during your own test run

map walks URLs. Your suite walks states — post-login pages, modals, wizard steps — and already handles whatever auth your app demands. Arm it with IAQA_CAPTURE=1 and the contract is captured during the real run. This is the intended path for a multi-domain / federated-SSO app, and it never touches your credentials.

Playwright — one import at the top of a spec (or wrap your own extended test with withCapture):

import { test, expect } from '@ia-qa/self-healing/capture';

Cypress — one import in cypress/support/e2e.js, nothing to wire in setupNodeEvents:

import '@ia-qa/self-healing/cypress';

Selenium (Node) — you call it yourself, since there is nothing to hook:

const { capture } = require('@ia-qa/self-healing/selenium');
await driver.get('https://app.example.com/checkout');
await capture(driver);          // inert unless IAQA_CAPTURE=1

All three are inert without IAQA_CAPTURE=1, so they can live in the suite permanently, and they never fail your tests. Each worker writes its own shard to .ia-qa/mapping/.capture/; the next verb (diff, run, baseline) merges them into the same page contracts map produces — byte-compatible, named from config.pages so they pair with your baseline.

Any other framework or language. The shard format is the interface; the helpers above are sugar over it. Run the extractor in the page (extract.js in this package, also a copy-paste DevTools snippet on ia-qa.com) and drop a JSON file into .ia-qa/mapping/.capture/:

{ "pages": [ { "page": "checkout", "url": "/checkout",
              "elements": [ { "role": "button", "name": "Pay", "selector": "button#pay" } ] } ] }

No import, no Node required. Coverage is whatever the suite visits — it is not a crawler, and a page it never reached is stale in the next diff, never a pass.

In CI

diff exits 0 on PASS/FIX, 1 on BLOCK, 2 on bad input:

# .github/workflows/selector-drift.yml
name: Selector drift
on: [pull_request]
jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm start &            # your app on its usual port
      - run: npx ia-qa-heal ingest  # inventory what your tests use
      - run: npx ia-qa-heal map     # re-capture into .ia-qa/mapping/
      - run: npx ia-qa-heal diff    # 0 = PASS/FIX · 1 = BLOCK · 2 = bad input

diff with no arguments compares the committed .ia-qa/baseline/ against what map just captured — which is why the baseline belongs in git. --strict fails on FIX too; --report drift.html attaches a branded report to the build.

Trend charts for free. diff --junit reports/drift.xml writes the verdict as JUnit XML, which Jenkins, GitLab, CircleCI, Azure DevOps and the GitHub Actions reporters ingest natively, per-build history included. It describes your contract, not your suite, so every language produces identically shaped XML. healable and BLOCK-class rows are failures typed by status, renamed passes with the label change in system-out, and a page that was not compared is skipped — never a pass.

⚠️ Publish it under its own file pattern, separate from your suite's results. Merged into one trend, both sets of numbers stop meaning anything.

--json on diff, run, audit, check and history gives the same verdict as one parseable document. Locally, ia-qa-heal history charts what no single run can reconstruct — and holds out runs captured a different way rather than averaging them in.

Configuration

.ia-qa/config.json, written by init. The minimum:

{
  "baseUrl": "https://staging.myapp.com",
  "pages": [
    { "name": "checkout", "url": "/checkout" },
    // a view with no URL of its own — reach it by naming the control to click
    { "name": "billing", "url": "/account",
      "steps": [ { "click": { "role": "tab", "name": "Billing" } } ] }
  ],
  "testPaths": ["tests/", "pages/"],
  "testCommand": "npx playwright test"
}

Your password never goes in this file. For a form login, secrets holds the name of an env var (or an SSM parameter path) and auth points at it by name:

"secrets": { "user": { "source": "env", "key": "APP_USER" },
             "pass": { "source": "env", "key": "APP_PASS" } },
"auth": { "loginUrl": "/login", "usernameSelector": "#email", "passwordSelector": "#password",
          "submitSelector": "button[type=submit]", "usernameSecret": "user", "passwordSecret": "pass",
          "successSelector": "nav.dashboard" }

Loading the env var is your job — the CLI reads process.env and never opens a .env file, so run it under npx dotenv-cli -- or Node's --env-file. If a variable is missing it stops and names it.

Everything else is optional:

--config <dir> (or IAQA_CONFIG_DIR) points at another .ia-qa/ — one per portal in a monorepo. Every field is documented in full in the tutorial (TUTORIAL.md, shipped in this package).

The console — ia-qa-heal ui

A local web console for the rows where the tool gave up: highlighted crops of each candidate, the file:line of every test that breaks, the verbs as plain-language buttons, and the history as charts. --shortcut writes a desktop launcher.

It is loopback only (127.0.0.1), every request carries a per-process token, the Host header is pinned, and its buttons spawn this package's own CLI with a fixed argv — never a string from the request. It refuses to start under CI (an interactive surface there is a hung build) and stops itself once the browser tab goes away.

Optional AI add-on — ia-qa-heal-ai (BYOK)

Deterministic healing leaves lost and ambiguous rows behind on purpose. The add-on hands only those to your own LLM and suggests a match with a confidence score:

npx ia-qa-heal-ai init            # pick a provider, name the env var holding your key
npx ia-qa-heal-ai suggest         # print suggestions
npx ia-qa-heal-ai suggest --apply # apply, after you confirm — refuses without a TTY

It never gates CI, and there is no --ai flag on ia-qa-heal: the verdict stays deterministic by design. Your key stays in your environment; the prompt carries the contract rows, nothing else.

What leaves your machine

Nothing, unless you enable the BYOK add-on above.

MCP server: path sandboxing. The CLI is driven by a human in their own shell; the MCP server receives arguments from an agent — potentially from third-party content via prompt injection. Every path parameter (out_dir, before_path, after_path, test_paths) is validated: the resolved absolute path must stay inside the project root. IAQA_ALLOW_OUTSIDE_PATHS=1 lifts it. browser_path is logged to stderr for auditability.

MCP server: URL allowlist. map_app verifies the target URL shares the origin of config.json's baseUrl, so an agent cannot be steered toward an internal-only endpoint reachable from your machine. IAQA_ALLOW_ANY_URL=1 for multi-origin apps.

Contracts are generated from live page content. The .md and .json files contain text extracted from the DOM — accessible names, headings, link destinations. The Markdown renderer escapes pipes, newlines and backticks to prevent table breakage and code-block injection. If you map an untrusted page (third-party widget, ad, XSS'd content), treat the output as untrusted before feeding it to an LLM: the tool does not detect or strip instruction-like text.

.ia-qa/session.json is a secret — live cookies. Gitignored on creation.

Going deeper

• The tutorial — the whole thing, step by step, no jargon: ia-qa.com/devtools/selector-drift/tutorial, or TUTORIAL.md inside this package.
• ia-qa-heal skill — installs this package's agent instructions into <project>/.claude/skills/, so an agent reads the contracts instead of opening a browser, and knows the rules it must not break. Agents read skills from .claude/skills/, never from node_modules, which is why this is a verb and not just a file in the tarball.
• ia-qa-heal-mcp — the stdio MCP server, for agents that prefer tool calls to a shell.
• SPEC-selector-resolution.md (shipped in the package) — how a Page Object's selectors are bound to the contract, and why each refusal exists.
• Peer dependencies — playwright is a peer: your project provides it, so two installs cannot drift to different browser revisions. @aws-sdk/client-ssm is an optional peer, needed only by the aws-ssm secret source (~5 MB, imported lazily; the CLI tells you if you pick it without).
• Browser reuse — it uses whatever Playwright browser is already on the machine before asking you to download 150 MB. IAQA_BROWSER_CHANNEL=chrome (or msedge), or IAQA_BROWSER_PATH=/path/to/chrome, to pin one.

MIT.