FerroGate is an open-source AI gateway that runs entirely on Cloudflare
Workers. It is a control point for AI traffic: OpenAI-compatible and
Anthropic-native inference APIs, an OpenAI-compatible Files + Batch surface,
multi-vendor provider routing with canary and shadow rollouts, virtual API keys
with scopes and (one SQLite-backed object
per tenant, addressed at runtime with no per-tenant deploy), policy and guardrail
screening, rate limits, quotas and prepaid wallets, durable token metering and
billing, an asset closed loop, an MCP server, agent runs, and a ~240-operation
admin API.
Open-source AI gateway on Cloudflare Workers, in TypeScript: OpenAI/Anthropic-compatible chat with SSE streaming, Files + Batch APIs, provider routing with circuit breaking and failover, virtual API keys, per-tenant Durable Object storage, quotas/wallets/budgets, guardrails, response cache, MCP server, durable metering, contract-driven Admin API.
Étoiles
29
Outils
—
Disponibilité
—
per-tenant Durable Object isolation
It is written in TypeScript end to end and deploys as a fleet of Workers backed
by D1, R2, KV, Durable Objects, Queues and Analytics Engine.
The project is developed as the open-source gateway foundation behind
Token4AI Cloud.
Architecture at a glance
Six deployables live under apps/. Five are Workers; the sixth is a CLI binary.
Deployable
Worker name
What it is
apps/gateway
ferrogate-gateway
The data plane. A Hono streaming proxy for inference, the OpenAI-compatible Files + Batch surface, and the asset surface. Defines the per-tenant TenantDataObject. Owns 49 contract operations.
apps/control-plane
ferrogate-control-plane
The admin API — ~240 contract operations (235 under /admin/v1/**, plus the /admin pages and /metrics), and the admin-console session surface, SAML, OIDC and SCIM.
Drain gate — 503 node_draining on spend-producing operations when the
fleet is drained.
Response cache — exact-match or opt-in semantic (in-tree feature-hashing
embedder and cosine similarity; no vector database), over the Cache API.
Zod validation, then the model registry (logical models, fallback,
canary and shadow splits).
Upstream dispatch — provider adapters with a circuit breaker Durable
Object, retry and failover; SSE framing is preserved byte-for-byte and a
client disconnect aborts the upstream.
Durable metering — the ledger row and the billing-outbox row commit in
the same D1 batch(), then publish onto a Queue.
After authentication, tenant-owned state is routed to a SQLite-backed Durable
Object addressed by tenant id. The shared control plane remains in one CONTROL
D1 database; this keeps tenant transactions isolated without provisioning or
deploying a database for each tenant.
Shared packages
Fifteen packages under packages/. Each exports src/*.ts directly — there is
no per-package build step.
Provider adapters — canonical plan in, upstream wire request out, normalized response/usage back.
routing
Route match plus deterministic canary/shadow rollout selection.
storage
The persistence boundary over D1/KV/R2.
billing
Rate cards, pricing, the idempotent ledger, outbox delivery.
payments
The x402 / Solana client-side wire contract (deprioritized).
observability
Logging, metrics and OTLP request construction.
cloudflare
Cloudflare account-management helpers for operator resources such as R2 buckets and scoped tokens.
sso
SAML 2.0 service provider.
identity
OIDC relying party and SCIM 2.0 provisioning.
Cloudflare products in use
D1 — the shared CONTROL database for account-wide configuration,
directories, compatibility data and cross-tenant projections.
R2 — asset object storage.
KV — MCP OAuth state.
Durable Objects — per-tenant SQLite storage plus rate limiter, provider
circuit breaker and shadow budget (gateway); MCP OAuth flow claim and MCP
session (mcp); agent run state and worker plane (agent-runtime).
Queues — the billing-report outbox producer.
Cache API — the response cache.
Analytics Engine — the telemetry sink.
Service bindings — gateway → telemetry.
Secrets Store — cf:// secret references, bound at deploy time.
Workers AI — the Llama Guard guardrail detector adapter. The [ai]
binding is supplied at deploy time; it is not declared in the committed
configuration.
The route contract
docs/openapi/runtime-api-contract.json is the authoritative source for the
runtime surface: 312 operations, each carrying path, method,
operation_id, visibility, auth.kind, auth.scope and rbac_action.
Every Worker imports it directly rather than restating it, and each app's
contract test fails if an operation it owns is not registered.
The split is 236 admin, 69 public and 7 internal operations; auth kinds are 298
bearer, 6 internal (worker-plane callbacks), 7 anonymous and 1
method-dependent. docs/rewrite/ROUTE-MAP.md assigns each operation to a
Worker. Field-level request and response bodies for the admin surface are in
docs/openapi/admin-api.openapi.json.
Clients generated from the contract
The TypeScript contract types, the Python operation catalog, and the
admin-console contract types are generated from
docs/openapi/admin-api.openapi.json and committed. The vendored SDK package
entry points are sdks/typescript/ and sdks/python/; every generated
artifact is listed in tools/generated-clients/artifacts.mjs and regenerated
by ONE command:
bun run generate # rewrites every generated client; commit the diff
Build and inspect the vendored package artifacts without publishing them:
bun run check:sdk-packages
Stale generated clients still compile — they simply describe an older server —
so a contract change that lands without regenerating goes unnoticed until
someone hits a type that is not there. Three did in one day (#676, #736, #737),
so bun run test fails on any client that no longer matches the document, via
tools/generated-clients/. That gate deliberately does not regenerate for you:
the regenerated diff is how a reviewer sees an operation appear, and a check
that quietly rewrote the artifact would make the contract change invisible.
Capabilities deliberately not offered
Three operations are mounted, guarded, and then refused with
501 capability_not_offered:
POST /v1/functions/execute
GET /v1/tools and POST /v1/tools/execute
This is a product decision, not an outage, not unfinished work, and not a
platform limit — nothing in the backlog tracks it and it is not a bug. The
decision, its reasoning, and what a re-implementer would need are recorded in
docs/rewrite/DROPPED-CAPABILITIES.md,
and a test hard-codes the dropped set so the refusal cannot be softened without
recording a decision.
Getting started
Prerequisites: Bun (the version is pinned by
packageManager in package.json). Wrangler and every other tool arrive as a
dev dependency — no global installs, no Cloudflare account, and no network
access are needed for the offline workflow below.
bun install
Run the tests
bun run test # every workspace
bun run typecheck # tsc --noEmit, every workspace
bun run lint # biome
bun run generate # regenerate every client generated from the OpenAPI contract
bun run test fans out to each workspace's own test script, and that
matters: four workspaces chain a second (and apps/gateway a third) Vitest
run behind a non-default config — apps/gateway (rate-limit and tenancy
harnesses), apps/agent-runtime (durable harness), packages/storage (D1) and
packages/routing (Durable Objects). A bare vitest run at the repo root or
inside one of those workspaces silently under-reports.
To run one workspace:
bun run --filter '@ferrogate/app-gateway' test
Run a Worker locally
wrangler dev --local boots the real workerd against local D1/KV/R2/DO
state. Apply the migrations first — wrangler dev provisions an empty SQLite
file per database id and does not run migrations_dir, and the gateway
correctly refuses to serve an empty schema:
cd apps/gateway
bunx wrangler d1 execute DB --local -y --file=../../sql/d1-ts/tenant/0001_init_tenant.sql
bunx wrangler d1 execute BILLING_DB --local -y --file=../../sql/d1-ts/control/0001_init_control.sql
bunx wrangler dev --local --ip 127.0.0.1 --port 8787
Each app also has bun run dev (wrangler dev) and bun run deploy
(wrangler deploy).
The committed [vars] are the fail-closed empties: with no credential
configured, every authenticated route answers 401 before its handler runs,
and with no provider or model configured the registry is empty and every model
answers 400 model_not_found. Override them for a local session with --var
(the end-to-end harness does exactly this) or a gitignored .dev.vars.
End-to-end
bun run test:e2e
Playwright starts a real wrangler dev per app from that app's production
wrangler.toml, applies the local D1 migrations, and drives the Workers over
HTTP. There is no browser — every spec uses the request fixture. A cold
wrangler dev takes 35–50s per app, so leave one running and the suite
attaches to it.
Deploying
Wrangler is the only bundler and the only deploy tool. There is no separate
build step: wrangler deploy bundles src/worker.ts per app.
Read docs/rewrite/CLOUD-VERIFICATION.md
before the first deploy. It is the ordered runbook, and the order is not
arbitrary — a service binding is resolved by name at deploy time, so
ferrogate-telemetry must exist before ferrogate-gateway deploys, and the
cross-Worker rate-limit bindings must be attached after it. That document also
enumerates the preconditions the repository deliberately does not commit:
Every database_id, bucket name, queue name and KV namespace id in
apps/*/wrangler.toml is a placeholder. No real account id, database uuid or
secret is committed.
D1 migrations (wrangler d1 migrations apply) must be applied before the
first authenticated request.
Secrets go in with wrangler secret put — the admin-console JWT secret, and
one per tenant SSO env:// reference.
Several committed [vars] are dev-posture defaults that must be overridden
in the deploy environment rather than flipped in the committed file, because
the offline suites drive the apps through them.
Durable Objects, Queues and Analytics Engine require a paid Cloudflare plan.
Repository layout
apps/ 6 deployables — 5 Workers + the CLI binary
packages/ 15 shared TypeScript libraries (source-only, no build step)
e2e/ Playwright black-box suite over real `wrangler dev`
sql/d1-ts/ D1 migrations: control/ and tenant/
docs/openapi/ the route contract + the admin OpenAPI document
docs/rewrite/ architecture, testing, deploy and parity records
Roughly 90k lines of TypeScript source and 114k lines of tests: 7,051 tests
across 385 files in 21 workspaces, plus 22 Playwright end-to-end tests.
Unit and integration — Vitest on @cloudflare/vitest-pool-workers,
which boots the real local workerd. D1, KV, R2 and Durable Object bindings
genuinely work; they are not mocked. Integration tests dispatch through
SELF from cloudflare:test.
Upstream mocking — MSW intercepts the gateway's outbound fetch() to
provider hosts and returns canned SSE, so token counting, stream
normalization and MCP forwarding are exercised deterministically. No real
LLM is ever called.
End-to-end — Playwright over wrangler dev, which is the only layer
that exercises Wrangler's own bundle and workerd's service registration —
a Worker can be correct under SELF.fetch and still fail to start as a
service.
Documentation
The current architecture is documented under docs/rewrite/:
Verifying the audit trail has not been altered:
audit-tamper-evidence.md — the hash chain,
the R2 anchor, the digest format, and scripts/verify-audit-chain.mjs with
the exit codes to wire into a job.
Getting request and audit logs into your own SIEM:
siem-export.md — the export pump, its at-least-once
guarantee and the two ways a row could still be missed, how to configure a
Splunk/Datadog/HTTPS/R2 sink without putting a credential in git, and how to
replay a window.
The API contracts are in docs/openapi/. Other documents under docs/ predate
the TypeScript implementation and describe the earlier system; treat
docs/rewrite/ and the contracts as authoritative where they disagree.
Contributing
FerroGate is built for human maintainers and AI coding agents working together.
The best contributions are small, issue-linked slices that can be reviewed,
tested, and explained from the operator's point of view. Day-to-day development
is organized around GitHub milestones, each grouping the issues that make up a
shippable increment.
Two conventions are worth knowing even for a one-off human patch:
Say what you did not verify. Commits carry Tested: and Not-tested:
trailers, and a handoff that reads as verified when it is not costs a whole
review round.
Assert the behaviour, not the call. A test that proves a guard was
invoked while the guard itself could be inverted with the suite still green
is the single most common defect this project has had to reject. If you add a
guard, name the mutation that would break it.
Practical rules:
Keep behaviour in the owning package; avoid cross-cutting rewrites.
Ship tests in the same change as the logic.
A new runtime route needs a contract entry — the contract is imported, not
restated, so a handler without one is unreachable and a contract entry
without a handler fails its app's test.
Include exact verification commands and known gaps in the PR.
Security
See SECURITY.md for the vulnerability-disclosure process.
Report suspected vulnerabilities privately; do not open a public issue.
Project history
FerroGate was previously implemented in Rust on Cloudflare Pingora. That
implementation was replaced by this one and is preserved at the git tag
legacy-rs.
License
Licensed under the Apache License, Version 2.0. See LICENSE.
F
Ferrogate
lianluo-esign·29 étoiles·
Open-source AI gateway on Cloudflare Workers, in TypeScript: OpenAI/Anthropic-compatible chat with SSE streaming, Files + Batch APIs, provider routing with circuit breaking and failover, virtual API k
Open-source AI gateway on Cloudflare Workers, in TypeScript: OpenAI/Anthropic-compatible chat with SSE streaming, Files + Batch APIs, provider routing with circuit breaking and failover, virtual API keys, per-tenant Durable Object storage, quotas/wallets/budgets, guardrails, response cache, MCP server, durable metering, contract-driven Admin API.
Outils
Outils
Pas encore de liste d'outils. Les serveurs distants en obtiennent une après le premier contrôle d'état.
FAQ
Questions
Comment installer Ferrogate ?
Copiez sur cette page la configuration stdio pour Cursor, Claude Desktop, Claude Code ou VS Code. Elle utilise l'installation de l'éditeur, pas un proxy PlugMCP.