five46
Say what should work, in plain English. An AI agent checks it against your real app or API, using your own LLM key, and hands you a real Playwright test.

You just built something and want to know whether it actually works, before
writing a test. Give five46 a goal like "log in and confirm the dashboard
loads" or "create a user via POST, then confirm it via GET". An LLM
(OpenAI, Anthropic, Gemini, Groq or AWS Bedrock, with your key) drives your
real app or API one action at a time. It tells you honestly whether the goal
was met, with a root-cause hypothesis when it wasn't. A successful run is
saved as a standalone Playwright (or node:test) spec, so today's check
becomes tomorrow's regression test, and running it needs no five46 or LLM.
Quick start
npm install -g five46
npx playwright install chromium # one-time browser download
five46 config # pick a provider and paste a key (Gemini's is free)
five46 doctor # confirms the key works
five46 test http://localhost:3000 --goal "log in as [email protected] and confirm the dashboard says Welcome back"
You get a pass or fail (and a matching exit code), a report with
screenshots and, for failures, a root-cause hypothesis, plus a
five46-agent-*.spec.ts you can commit and run with npx playwright test.
Testing an API instead? Use five46 api <base-url> --goal "...". It's
read-only unless you pass --allow-writes.
Status: early and actively developed. Verified end-to-end against dozens of real-world sites and APIs.
Check every pull request
The five46 GitHub Action reads the Acceptance criteria section of a pull request's description and checks each item against the PR's preview deployment. It keeps one PR comment up to date with the verdict per criterion, why each failure failed, and a ready-to-commit spec for every criterion that passed.
on: deployment_status # Vercel, Netlify and other preview deploys
jobs:
five46:
if: github.event.deployment_status.state == 'success'
runs-on: ubuntu-latest
permissions: { contents: read, pull-requests: write }
steps:
- uses: sekharsdet/five46-action@v1
with:
url: ${{ github.event.deployment_status.target_url }}
api-key: ${{ secrets.FIVE46_API_KEY }}
How it compares
Already working in an AI coding tool and growing a Playwright suite there? Playwright's own agents are a great fit, especially the healer for keeping existing tests green. five46 is for getting a verdict: is this feature working right now, in CI or on a PR, for a browser flow or an API, with any LLM you already pay for. More detail is in Why five46.
Highlights
- Safe by default. API runs are read-only unless you unlock writes; destructive-looking clicks ("Delete account") are blocked unless allowed.
- Login credentials never reach the LLM. It sees placeholders, and the real values are filled in locally. Log in once, then reuse the session.
- Acceptance criteria in, verdicts out.
--storysplits a user story or ticket into independent scenarios and reports pass/fail per criterion. - Specs you can trust.
--verify-clean-sessionre-runs the spec cold;--verifymutates its assertions and requires them to fail. - Works with any current key. If your provider retires a model, five46
finds a current one your key can use. You can also pin one with
FIVE46_LLM_MODEL. - Fits your tools. CI-friendly exit codes,
--reportJSON for dashboards, flaky-run detection (--repeat), video recording, and an MCP server for IDE assistants.
Docs
- Configuration: providers, getting a key,
five46 doctor, choosing models - CLI reference: every command and flag, login sessions, API testing, story mode, verification, reports
- MCP server: use five46 from an IDE-embedded AI assistant
- Security model: what leaves your machine, what five46 guards, and what's your call
- Why five46: design choices, the full feature list, comparisons
- CHANGELOG
Known limitations
- No closed shadow DOM traversal. Elements inside a
mode: "closed"shadow root aren't visible to the agent's page snapshot (a closed shadow root is deliberately inaccessible to external tooling in general, not just five46). Open shadow roots, the overwhelming majority of real-world web components, work fine, as do<iframe>and<frame>content. - Chromium only for browser mode, with no Firefox or WebKit yet.
- A run itself is not deterministic. The same goal against the same page can take a different path next time. The generated spec is the frozen, repeatable artifact.
Contributing
Issues and pull requests are welcome. See CONTRIBUTING.md.
git clone https://github.com/sekharsdet/five46.git
cd five46
npm install
npm run build
node dist/cli.js test <url> --goal "..."
npm test
If five46 is useful to you, a ⭐ on GitHub helps other people find it.