@codewithjuber/forgekit

Forgekit — reliability infrastructure for AI coding agents

Your coding agent forgets. Forgekit remembers — outside the model, in plain git.

Forgekit is a Node.js CLI and MCP server (zero runtime dependencies) that gives your AI coding tools a reliability layer: agent memory that survives sessions and teammates, pre-push verification gates, and guardrails that run before and after every tool call. You author project rules once, and it emits native configuration for 12 agent tools. Claude Code is the most deeply exercised integration.

Why it wins, measured:

• Hooks stay cheap. SessionStart injects a pointer index, not the full memory — measured 18,262 B of durable memory → 826 B index — and every hook event has a hard byte budget (hooks/budget.json: 12 KB SessionStart, 4–6 KB the rest), enforced by the launcher. No silent context bloat.
• Verification that can actually fail the push. forge verify runs the repo's real test suites and reports PASS / FAIL / INCOMPLETE / NOT_CONFIGURED against a tree fingerprint of the code it ran on — a red run blocks the push, not just the chat.
• Blast radius before you edit. forge impact lists the files a change probably touches from a cached code graph: 1.68 ms warm (the cache is built by forge init).

It does three jobs:

1. Share reliable project knowledge. Lessons, facts and decisions live in the repository as evidence-referenced memory: each claim carries references to its evidence, only tests, CI or a person raise its confidence, and teammates merge it over plain git.
2. Inspect likely change impact. Before an edit, a heuristic impact analysis lists the files the change probably touches, including coupled files you did not name. It reads a regex-derived code graph, so it can miss files as well as over-warn.
3. Verify the current change. forge verify runs the repository's detected test suites and reports PASS, FAIL, INCOMPLETE or NOT_CONFIGURED for the code state it ran on. A run that covers only part of the repository is partial verification, and says so.

Status: beta — read before you rely on it.

• Releases follow Semantic Versioning (CHANGELOG, release notes): since 1.0.0 a breaking change ships as a new major version. "Beta" describes maturity, not interface churn — the analyses are heuristic, the checks are advisory unless you enable enforcement, and most integrations have less real-world exercise than Claude Code. The core (init, sync, substrate, impact, ledger, guards) is tested and in daily use.
• Claude Code is the deepest-tested integration (full plugin, ambient UserPromptSubmit guards). The other eleven tools receive native config (most also an MCP server entry; Aider, Windsurf/Devin and Kimi Code do not) but no automatic hooks, and have had less real-world exercise. On OpenClaw specifically, rules arrive via AGENTS.md project context; the config-only path uses a one-command MCP registration, while installing the package as a compatible Codex bundle loads its skills and bundle-scoped MCP server. Neither path provides ambient hooks (see OpenClaw in ARCHITECTURE).
• Impact/blast-radius analysis is heuristic — a regex-approximate code graph, not a sound call graph. It is not conservative: it can miss affected files as well as flag unaffected ones, so treat its output as advisory and an empty result as "unknown", not "safe".
• "Proof-carrying memory" is a name, not a formal proof. Claims are content-addressed and carry evidence references; confidence moves only when independent oracles (tests, CI, a human) raise it. There is no theorem-prover in the loop.
• Some integrations shell out — forge harden, forge scan, and the git-native ledger assume Bash and Git are available (jq is not required — the guards read hook JSON with node). Claude hooks on Windows do not require bash on PATH: their Node launcher finds Git Bash and preserves guard exits. protect-paths needs no bash at all and fails closed: when it cannot reach a verdict, the tool call is blocked rather than let through.
• Guardrails reduce risk; they are not a security sandbox. What each integration actually does — config emission, MCP registration, automatic hooks, blocking — is in docs/INTEGRATIONS.md.

60-second quickstart

npm install -g @codewithjuber/forgekit   # or: npm install -g github:CodeWithJuber/forgekit
forge init                               # emit native config for the tools this repo uses
forge doctor                             # verify providers, hooks, and MCP wiring
forge demo                               # watch the gate catch a real defect (green → broken → green)

Then run a pre-action check inside a repository:

forge substrate "Change verifyToken in src/auth.js to require length > 20; update tests"

The result includes an assumption verdict, a model-tier recommendation, predicted impact, a context-coverage check (what was delivered, not a proof that it suffices), scope clusters, and a verification checklist. On Claude Code, the pre-action check can run automatically through a UserPromptSubmit hook. On other supported tools, Forgekit emits instructions and MCP configuration that the tool can invoke.

The three jobs, one command each:

forge ledger query "token validation"   # 1. what does the team already know about this area?
forge impact verifyToken                 # 2. which files will this edit probably touch?
forge verify                             # 3. did the change pass the suites that actually ran?

See it in action

Real terminal recordings of the actual CLI — nothing mocked:

forge verify on a change that imports a package that doesn't exist: it flags the hallucinated dependency (frobnicator — not in the codebase) and blocks the push.

A fake secret staged with git add: forge precommit spots the credential-like line and refuses the commit before it ever happens.

Cross-session memory: forge recall add saves a durable fact (deploy-window), forge recall list reads it back — memory that survives across agent sessions.

Contents

• 60-second quickstart
• See it in action
• Why Forgekit exists
• How the loop works
• Core capabilities
• Agent orchestration
• Advisor pairing
• Setup details
• Commands
• Team memory
• LLMs, retrieval, and embeddings
• Agent roles and MCP tools
• Honest limits
• Portfolio evidence
• Maturity boundary
• Measured evidence
• Structural comparison
• Python research prototypes
• White paper
• Public site
• Documentation
• Your next step
• Community and support

Why Forgekit exists

The pain it kills — and the mechanism that kills it:

• You fixed this bug last month. The agent just re-introduced it. Lessons die when a session compacts. Forgekit stores them in the repository as evidence-referenced memory: only tests, CI, or a human raise a claim's confidence, and wrong lessons decay out instead of ossifying. forge ledger query "token validation" finds what the team already knows — teammates get it by pulling. (Memory persists outside the model; it doesn't guarantee the agent reads it — retrieval is budget-bounded, and missing context becomes a question, not invented context.)
• The "safe" rename broke twelve files you never opened. forge impact <symbol> shows the predicted blast radius before you edit, including coupled files you never named. It is heuristic — a regex-derived graph, not a sound call graph — so treat the output as advisory and an empty result as "unknown", not "safe".
• "Done" with no test evidence. A completion gate blocks a session's "done" when code moved without test evidence or a doc/state artifact — and hands back the repair checklist (forge docs sync sweeps the diff for stale prose, forge handoff writes the bounded session snapshot, forge decide records choices so no session re-decides them). On Claude Code this runs automatically.
• Rules that die at context compaction. Guardrails — protected paths, cost budget, doom-loop detection, secret redaction — run as deterministic hooks, not prose in a prompt. They survive compaction the way CLAUDE.md does not. (Ambient hooks are the Claude Code integration; other tools get config and instructions. The substrate is advisory by default; set FORGE_ENFORCE=1 to block only its strongest signals.)
• One rulebook, eleven tools. Author your rules once; Forge emits each tool's native config, plus MCP for Roo and VS Code. Zero runtime dependencies — one Node CLI, plain files in git, no server to run.

How the loop works

Forgekit runs a deterministic substrate before work, lets the external coding agent act, and records evidence from tests, CI, or explicit human correction afterwards.

Open the interactive diagram: pan, zoom, search, and trace any node.

Only independent oracles (tests, CI, a human accept/revert) move a memory's confidence — so a wrong lesson decays out instead of ossifying. Full design: ARCHITECTURE.md.

What you get

The day-to-day value first — what the substrate keeps outside the model, so it survives between sessions and tools:

• Memory that persists across sessions and teammates. [Implemented] Every lesson, fact, and verified reuse is proof-carrying memory (PCM) — our name for evidence-referenced, content-addressed memory: a claim that carries references to its own evidence and is only trusted once independent oracles raise its confidence above a floor (the "proof" is that evidence trail, not a formal proof). Wrong lessons decay out instead of ossifying.
• Heuristic impact before you break things. [Heuristic] Ask "what does changing verifyToken break?" and get the blast radius — the set of files an edit is predicted to impact, read from a regex-approximate code graph (not sound, and it can miss affected files), including coupled files you never named.
• Guardrails that can't be forgotten. [Implemented on Claude Code] Deterministic hooks check the rules a model shouldn't break (protected paths, cost budget, doom loops) — they survive a context compaction the way CLAUDE.md prose does not. They reduce risk as defence in depth; they are not a sandbox, and a sufficiently creative shell command can still bypass a regex guard.
• Work that finishes end to end. A completion gate blocks "done" once per session when code moved without test evidence or a doc/state artifact (a UI-only change — styling, class strings, JSX text — owes a design record or a UI check instead of a unit test, and in a shared checkout a file another agent's session trail claims is named, not weighed; anything no trail accounts for still counts) — with the repair checklist as the answer (forge docs sync sweeps the diff for stale prose, forge handoff writes the bounded session snapshot the next session resumes from, forge decide records choices so no session re-decides them).
• One config for 11 tools. Author your rules once; Forge emits each tool's native config, plus MCP for Roo and VS Code. Zero runtime dependencies — one Node CLI, plain files in git, no server.

The measured evidence

Every number is a median from npm run bench on this repo, recorded with its environment block in reports/benchmarks.md — the project rule is a number is an assumption until measured.

• Blast radius in 1.68 ms (warm code-graph). On 6 hand-labeled cases from this repo's real import graph, recall is 1.00 against 0.26 for looking at the edited file alone, and precision is 0.17 — impact walks reverse dependencies transitively by default, so it returns everything downstream while the labels name only the direct referencers (restricted to one hop the same cases return their labeled sets). The precision 0.90 this line used to quote does not reproduce. On nine real Python repositories the research prototype's impact oracle reached recall 0.022 (refutation).
• A full pre-action gate in 851 ms (median on this repo, warm, on a 4-core Linux VM — this row is machine-bound; see the environment block) — assumption check, routing, reuse lookup, context assembly, blast radius, scope, and goal anchor in one deterministic pass, no LLM call. On Claude Code it runs on every prompt, automatically.
• The white paper's 62.1% routing saving is refuted. It was measured on the 30 tasks the prototype's thresholds were tuned on. On 80 held-out tasks from real issues, counting every escalation, the pipeline spent 20.2% more than always using the premium tier; per output a judge accepted it cost $1.06 against $1.76, but only 6 and 3 of 64 outputs were accepted (refutation). forge cost --stages reports only your measured stages.
• Conflict-free team memory — merging two 500-claim ledger replicas takes 191 ms on that same VM (I/O-bound, so it varies with the disk); the merge is order-independent and property-tested, so teammate ledgers converge to the same state no matter who syncs first, over plain git. The substrate is advisory by default. Set FORGE_ENFORCE=1 to block only its strongest signals: a vacuous task, required context that cannot be assembled, or a large impact set from a fresh repository graph.

Core capabilities

• Evidence-weighted memory. Claims are content-addressed, keep provenance and outcome references, and earn or lose confidence through a closed set of oracles. Time decay moves unreviewed knowledge toward uncertainty rather than silently treating it as permanent fact.
• Git-native team merge. Claims and append-only logs merge by set union. The join is property-tested for commutativity, associativity, and idempotence.
• Heuristic impact prediction. Forgekit builds a regex-derived code graph and walks reverse dependencies to estimate affected files and tests; the pre-action check and the Stop gate's repair checklist also walk the empirical refutation's sibling and forward relations and tag each file with the relation that reached it. It is not conservative: it can miss affected files (including constructs its parser does not recognize) as well as produce false positives, and the sibling/forward files are lower-precision co-change candidates.
• Budgeted context assembly. Definitions, direct dependants, sibling tests, and trusted lessons are selected under a token budget. Missing required context becomes a question rather than invented context. Coverage is syntactic — delivered, not proven sufficient — token counts are estimates, a file that only fits as a "read this" pointer stays a pending read, and a definition counts as delivered only when its whole body is.
• Model-tier recommendation. A deterministic rubric combines task text and repository signals. An optional LLM proposal can only lower the tier, confidence-gated and bounded; a vote for a higher tier is never applied automatically — it is recorded as an advisory escalateTo recommendation, which names the tier only once an external check has actually failed (the doom-loop diagnosis at its thrash threshold). Forgekit advises which tier to request; it does not itself proxy or fail over model traffic. The opt-in cross-provider router (route universal) minimises expected routing cost for a success target; its budgets are expectations, not spend caps, and its benchmark headline is repository-reported. Its shipped prior refits exactly from pinned public data in this repository, and a new in-repo held-out replay finds it no better than a fixed cascade chosen on the same dev tasks (docs/UNIVERSAL_ROUTING.md). For agents nobody watches, --mode unattended adds a confidence-gated vote raise, a writes-code floor, a risk floor and a gate on the top tier (Agent orchestration).
• Proof-gated reuse. Cached code is served only after evidence clears a confidence floor and declared dependencies still resolve in the current repository graph.
• Lifecycle guardrails. Claude Code hooks cover prompt preflight, protected paths, cost budget, repeated failures, format-on-edit, secret redaction, completion checks, and session learning. These controls reduce risk; they do not create a secure execution boundary.
• Verification, partial or full. forge verify runs the repository's detected test suites, reports PASS, FAIL, INCOMPLETE, or NOT_CONFIGURED, checks unknown symbols, and binds provenance to the code state it ran on. A verdict covers only the suites that ran: partial coverage is INCOMPLETE, not PASS. --deep adds structural, security, spec-drift, impact, and optional model-review lenses.

Agent orchestration

A lead agent running several sub-agents mostly pays for context re-reads, not new work. In one measured session, 7 parallel agents on one shared e2e environment used ~857M tokens in ~4.5 h, ~98% of them cache re-reads: agents polled logs every ~10 minutes against a 250–300k-token context, more agents ran than the environment could serve, and the top model tier was hand-picked for routine work (~140M tokens).

forge sync now emits an orchestration rule pack into every tool's config: no polling, a parallel-agent cap (default 2), heavy suites in CI, fresh small sub-agents with short reports, model choice via forge route, tests plus lead review for money/auth/secrets/migrations/security changes, isolated e2e environments, and a lean lead. Each rule has a stable id (orch.no-polling, …) and can be switched per repo or globally:

forge orchestration                        # rules, routing settings, and where each value came from
forge orchestration off orch.parallel-cap  # this repo; add --global for every repo
forge orchestration set route.mode unattended
forge route "add a refund endpoint" --mode unattended --json   # tier key + policy steps

In unattended mode, forge route raises the tier on a confident proposer vote, starts code-writing tasks at mid, keeps risky tasks on the premium tier, and allows the top tier only with --allow-top and a top deterministic score. The default conservative mode is unchanged. Details, defaults and the evidence: docs/ORCHESTRATION.md.

Advisor pairing

Claude Code's advisor tool lets the main model consult a second, typically stronger model at decision points: before committing to an approach, when an error keeps recurring, and before declaring a task done. The advisor reads the whole conversation and is billed at its own rate per call, and Claude Code accepts it only when it ranks at or above the session's main model. forge advisor configures it once, per repo or globally, emits it into Claude Code's settings, and checks the pairing before Claude Code has to refuse it:

forge advisor set opus                     # this repo: .forge/forge.config.json → .claude/settings.json + an Advisor section in AGENTS.md
forge advisor set fable --global           # every repo: the user-level config → ~/.claude/settings.json (what /advisor saves to)
forge advisor check sonnet opus            # ok — exit 1 when the pairing would be refused (for scripts and CI)
forge advisor pairings claude-opus-4-8     # which advisors this main model accepts, and the suggested one
forge advisor off                          # remove advisorModel again

The Advisor section tells every agent that reads AGENTS.md when to consult it (approach, recurring error, done check, risky code), when not to (short, well-specified tasks), and that the evidence wins over its guidance; each rule has a stable id (advisor.before-approach, …) and can be switched with forge advisor rule off <id>. The prompt-time advisory adds one line at a decision point, the doom-loop advisory names the advisor when a failure recurs, and forge doctor reports the pairing and the environment variables that keep the advisor off (CLAUDE_CODE_DISABLE_ADVISOR_TOOL, DISABLE_TELEMETRY, CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC). The pairing table ships as data (source/advisor.json, with the date it was verified against the docs); a model newer than it is accepted with a note, since Claude Code validates the pairing at launch.

Setup details

Install using one path:

The plugin runs its guards and its forge-cortex MCP server from the plugin directory, so it needs only node on PATH. It does not put the forge command on your shell's PATH; to run the forge commands below yourself, also install the npm package.

Initialize inside a project:

forge init
forge doctor
forge doctor --fix

forge init can merge Forgekit hooks and permissions into ~/.claude/settings.json. That file is global and affects all repositories. Use forge init --no-settings to skip the merge or forge init --remove-settings to reverse Forgekit-managed entries. The implementation preserves unrelated entries and creates a timestamped backup before changing the file. When the Forgekit Claude Code plugin is enabled (in user, project or local settings), its hooks/hooks.json already runs every guard, so forge init and forge doctor --fix merge permissions only and never register the guards a second time.

forge init emits configuration only for the agent tools the repository already uses: Claude Code plus any tool with a sign on disk (.cursor/, .codex/, .github/copilot-instructions.md, and so on). Choose explicitly with forge init --tools claude,cursor or --tools all; the choice is recorded in .forge/forge.config.json, so later forge sync runs emit the same set. In AGENTS.md, Forgekit owns only the block between and. A hand-written AGENTS.md keeps every line and gets the block appended; forge sync and the Stop-hook auto-sync rewrite that block and nothing else.

For an explicit model provider, inspect or update configuration with forge config. API keys remain environment variables; Forgekit's provider file stores the environment-variable name, not the secret value.

Commands

Commands are advisory unless their documented enforcement flag is enabled. Full worked examples and output live in docs/GUIDE.md.

Terminal output is plain when piped. On an interactive terminal it can use color and confidence meters; NO_COLOR disables color and FORCE_COLOR=1 enables it explicitly.

Team memory

Lessons, durable facts, and verified reuse artifacts land as content-addressed claims under .forge/ledger/:

forge init

# Work normally; hooks and explicit memory commands add claims and evidence.
git pull
forge ledger merge <path-to-another-ledger>
forge ledger sync

Identical claim content converges to one identifier while provenance records preserve authors. forge ledger blame <id> shows the mint history, oracle outcomes, and derived trust. The default storage is files in git, not a hosted database or synchronization service.

With no flags, forge ledger sync serializes ledger state under refs/forge/ledger on the repository's git remote. A non-fast-forward race triggers a re-merge and bounded retry. A shared directory can be selected with --dir <path> or FORGE_SYNC_DIR; --personal includes the per-user ledger.

LLMs, retrieval, and embeddings

Direct LLM adapters

src/llm.js supports two wire formats:

• Anthropic Messages API;
• OpenAI-compatible chat completions, used for OpenAI, Gemini, OpenRouter, and LiteLLM.

Credentials are read from environment variables and are not put in command-line arguments. The implementation performs a bounded single-prompt request. It does not implement streaming, conversation persistence, or a model-driven tool-call execution loop.

Repository-local retrieval

Forgekit retrieves from its code graph and evidence ledger, then assembles a context bundle for an external coding agent. This is retrieval and augmentation for source-code work, not a general enterprise RAG pipeline. There is no document connector layer, chunking service, citation generator, managed index, or vector database in this repository.

Optional embeddings

MinHash remains the zero-dependency default. To opt into semantic similarity, configure an external provider with one of the formats the current implementation accepts:

# OpenAI-compatible embedding endpoint
export FORGE_EMBED="https://api.example.com/v1/embeddings"
export FORGE_EMBED_MODEL="your-embedding-model"
export FORGE_EMBED_KEY="your-provider-key"

# Or a local/external command that implements Forgekit's stdin/stdout vector protocol
export FORGE_EMBED="cmd:./my-embedding-provider"

Vectors are cached in .forge/embed-cache.jsonl. Provider failure, timeout, malformed output, or missing vectors falls back to MinHash. The repository test uses a deterministic fake provider; it verifies adapter and fallback behavior, not live performance of a hosted model.

The performance snapshot in reports/benchmarks.md was generated at commit eb68ea9 and does not measure the optional embedding path. Current embedding behavior is defined by src/embed.js and test/embed.test.js.

Agent roles and MCP tools

Forgekit registers five Claude Code role definitions:

• scout — read-only repository investigation;
• verifier — fresh-context correctness review;
• independent-reviewer — diff/spec/test-only merge gate;
• frontend-verifier — visual and accessibility review;
• doc-sync — documentation consistency after code changes.

These are concrete role prompts supplied to the host tool. Forgekit does not contain a runtime that spawns these roles, exchanges inter-agent messages, or executes a LangGraph-style state machine.

The built-in MCP server exposes 21 tools, including:

• pre-action checks: substrate_check, preflight_check, assumption_gate, route_task;
• repository analysis: predict_impact, scope_files, rank_code, collide_check;
• memory: cortex_lessons, forge_remember, forge_ledger_query, forge_ledger_ratify, forge_ledger_retract;
• operations: forge_doctor, forge_provider_status, forge_cost, dashboard data.

The MCP server is the tool-provider side of function calling. The compatible host remains responsible for deciding when to call a tool and feeding the result back to its model.

Honest limits

• Beta software. The CLI is released, CI-tested and semantically versioned (a breaking change is a major version); "beta" is about maturity and evidence, not about interface churn. Support is maintainer-led and best-effort; there is no SLA.
• Claude Code is the deepest-tested integration. Other emitters and MCP configuration are implemented, but the repository does not provide equivalent real-world exercise evidence for every supported host.
• The code graph is heuristic. Regex extraction is not a sound call graph and can both over-predict and miss dependencies.
• Routing is advisory. Forgekit recommends a tier. LiteLLM or another gateway must be run separately to move traffic, handle failover, enforce quotas, or manage credentials. Routing costs are expected values, not caps, and no cost saving has been measured end to end: the ~90 % figure in the plans is a target (a hypothesis), and the white paper's 62.1 % routing saving was refuted on held-out tasks.
• Memory is external state, not model training. Claims live in files and are retrieved into context; Forgekit does not update model weights. It supplies persistence and external checks that a model does not guarantee on its own — one tested way to do that, not the only one.
• Embeddings are an adapter, not an included model or vector store. A user supplies the external command or HTTP service. MinHash remains the default and fallback.
• Guardrails are defence in depth. Shell-regex checks can be bypassed and post-tool redaction runs after the external tool has executed. Forgekit is not a sandbox.
• Human ratification is a workflow convention. Ratify and retract operations record the git author, but this repository does not authenticate a human or enforce enterprise approval roles.
• Evaluations are deliberately scoped. The JavaScript impact set has six hand-labelled cases; the Python router set has 30 hand-labelled tasks tuned to its rubric; the Python impact study has five mutations. None is a production field study.
• No enterprise platform claim. There is no Azure OpenAI/AI Foundry integration, vector DB, business-system/RPA connector layer, multi-tenant service, Kubernetes deployment, IaC stack, or public production-usage case study in this repository.

Portfolio evidence

The project is best read as agent reliability and developer tooling. It is not presented as an enterprise multi-agent application, a general-purpose RAG platform, or an Azure AI deployment. Per-claim status (implemented, measured, refuted, hypothesis, partial, reported) is kept in docs/status/.

For reviewers evaluating hands-on Agentic AI or GenAI work, each claim below links to the implementation and its closest test or build proof. The evidence snapshot used for this table is default-branch commit 3d9be37.

Maturity boundary

Personal maintainer use is intentionally not used as proof of organizational adoption. No customer count, enterprise deployment, production traffic, or service-level claim is made without corresponding public evidence.

Measured evidence

Numbers below are reported only with their test boundary. See reports/benchmarks.md and the linked evaluation artifacts for full methodology.

Parser-stable snapshot labels used by the generated project pages are:

• A full pre-action gate in 851 ms median — deterministic, warm repository graph, LLM disabled;
• Blast radius in 1.68 ms median — warm impact query; and
• 20.2% more cost than always-premium — the held-out routing result. The 62.1% saving the white paper reported came from a 30-task demonstration with thresholds tuned on those same tasks; on 80 pre-registered held-out tasks the same router spent 20.2% more (table below). Per judged-correct output the pipeline cost $1.06 against always-premium's $1.76.

The boundaries in the table below are part of each result.

Recent audited CI runs cover Node 20, Node 22, and Windows Git Bash, plus the reusable quality gate (CI workflow). The quality gate runs the Node unit suite, Biome checks, TypeScript type checking, critical-level npm audit, ShellCheck, zero-runtime-dependency assertion, version and documentation checks, and npm pack --dry-run. The Python prototype pytest suites were not part of CI at that commit; since aedddf5 a separate CI job runs both prototype suites, the Theorem D sanity checks (research/recompute_corrections.py --theorem-checks) and the recomputation of every corrected number from the replication package.

Structural comparison

This table describes architecture, not a claim of superiority or equivalent product scope.

Python research prototypes

The Python packages under research/python-prototypes/ are the original research baselines preserved for auditability and result reproduction. They are useful evidence of Python implementation work, but they are not the shipped Forgekit runtime.

Router Gate

router_gate implements:

1. deterministic assumption assessment;
2. transparent cheap/mid/premium routing;
3. a pluggable executor;
4. caller-supplied verification;
5. bounded escalation after failure;
6. token and cost records;
7. CLI and MCP surfaces.

Its checked-in evaluation used live model calls for a 30-task hand-labelled demonstration. The set was used to tune the rubric, so perfect separation is evidence that the mechanism works on that set, not a general accuracy estimate. Despite legacy package metadata using the phrase “production-ready,” the repository-level maturity classification is research prototype.

Impact Oracle

impact_oracle parses Python ASTs, stores a NetworkX dependency graph as JSON, and predicts change impact through reverse-dependency traversal plus the repaired sibling and forward relations (the in-tree package is the repaired v2, 49 tests). The bundled evaluation mutates five symbols and uses pytest failures as independent ground truth. On nine real repositories the as-shipped traversal reached recall 0.022; the repaired version's held-out point estimate is above grep's, but that it beats grep is not established (refutation). The Node forge impact graph is a different, regex-derived implementation and inherits none of these numbers.

The production Forgekit equivalents are the Node implementations behind forge preflight, forge route, forge atlas, forge impact, and the MCP substrate tools.

White paper

The cognitive-substrate white paper explains the motivation, formal model, evidence map, adjacent work, and the relationship between the Python research prototypes and the current Node implementation. Treat the paper's measurements according to their stated methodology; do not blend results from different codebases or evaluation sets. The "faculty" and theorem language lives there and in research/, with dated corrections: the papers now state what a frozen model does not guarantee (durable state across calls, context beyond the window, weight updates from outcomes, reliable self-verification) rather than claiming it cannot adapt at all, and they name the prior art (CoALA, Reflexion) the architecture builds on. Their PDFs are historical, pre-correction editions; read the HTML.

Public site

Forgekit includes a hand-authored landing page at landing/index.html and generates a status page at public/index.html. The generator reads repository data from package.json, README.md, CHANGELOG.md, and the benchmark report.

npm run pages:build
BUILD_PAGES_LIVE=1 npm run pages:build

The default build is offline and deterministic. Optional live mode reads public GitHub repository metadata with bounded retries and caching. GitHub Pages deployment is defined in .github/workflows/static.yml; this static documentation site is not an AI application deployment.

Documentation

Current implementation truth for optional embeddings is src/embed.js: use an http:// or https:// endpoint, or cmd:<command>. A bare FORGE_EMBED=1 is not a valid provider configuration. Benchmark comparison prose that describes MinHash-only behavior is historical and does not supersede the current source and tests.

Your next step

Sixty seconds, three commands, and your agent tooling gets a memory:

npm install -g @codewithjuber/forgekit
forge init      # emit native config for the tools this repo uses
forge doctor    # verify providers, hooks, and MCP wiring

Then try it on your own repo:

forge impact <symbol>     # what will this edit probably touch?
forge ledger query "..."  # what does the team already know about this area?
forge verify              # did the change pass the suites that actually ran?

• Found a gap or a wrong answer? Open an issue — the project's own rule is a number is an assumption until measured.
• Using it in your workflow? ⭐ Star the repo and tell us what it caught.

Community and support

• Get help: SUPPORT.md and Discussions
• Contribute: CONTRIBUTING.md and Code of Conduct
• Direction: ROADMAP.md and GOVERNANCE.md
• Security: SECURITY.md for private vulnerability reporting
• Accessibility: ACCESSIBILITY.md

MIT licensed. Built by CodeWithJuber.