FleetQ — Open-Source AI Agent Orchestration Platform
Self-hosted mission control for AI agents. Build, run, and monitor autonomous multi-agent systems with a visual DAG builder, human-in-the-loop approvals, MCP server integration, and full audit trail. Works with Claude, GPT-4o, Gemini, Ollama, Codex, Claude Code, and any OpenAI-compatible LLM.
Open-source AI agent orchestration platform — self-hosted mission control for autonomous multi-agent systems. Visual DAG workflows, 670+ MCP tools, human-in-the-loop approvals. Works with Claude, GPT-4o, Gemini, Ollama, Codex.
Sterne
81
Tools
—
Uptime
—
Keywords: AI agents · agent orchestration · MCP server · Model Context Protocol · LangGraph alternative · CrewAI alternative · n8n for AI · Claude agents · LLM workflow · autonomous agents · agent framework · AI automation · self-hosted
☁️ Prefer managed? Try FleetQ Cloud — zero setup, free tier.
⭐ Like the project? Give it a star on GitHub — it helps others find FleetQ.
Most agent frameworks give you a Python notebook. FleetQ gives you a production platform.
🧩 670+ MCP tools across 60+ domains — every feature is exposed via Model Context Protocol, so any LLM (Claude Desktop, Cursor, ChatGPT, local agents) can drive the platform programmatically. New in 1.27: web UIs for previously headless capabilities (agent sessions, release signing keys, drift & eval monitors, broadcasts, test suites, CSV import); eight outbound chat channels as first-class drivers; the Agentic AI Flywheel (self-growing eval set + drift/production monitors); policy-governed autonomy (versioned per-agent policies + replay); cost-aware orchestration and Return on Cognitive Spend (ROCS) metrics.
🔁 Visual DAG workflows with 8 node types (agent, conditional, human-task, switch, dynamic-fork, do-while, compensation, sub-workflow) — no Python glue code.
👥 Multi-agent crews with coordinator/worker/reviewer roles, weighted QA scoring, and cross-validation.
🛡️ Real-World Action governance — assistant tool calls, integration writes, and git pushes route through a per-tier risk policy (auto / ask / reject for low / medium / high). Approvals auto-execute. Audit trail attached.
💰 Budget controls with a real credit ledger, pessimistic locking, and auto-pause on overspend — not just token counters.
🧠 Agent evolution — LLM analyzes execution history and proposes config changes you approve with one click.
⚙️ BYOK + Local LLMs — Anthropic, OpenAI, Google, plus Ollama, LM Studio, vLLM, Codex, Claude Code. Zero vendor lock-in.
Trigger rules — event-driven automation with condition evaluator, dry-run testing
Multi-Channel Outbound — Email (SMTP), Webhook, ntfy plus eight chat channels as first-class drivers (Telegram, Slack, Discord, Microsoft Teams, Google Chat, Matrix, Signal, Supabase Realtime), each with a config page, rate limiting and blacklist
Marketplace — browse, publish, install shared skills, agents, workflows, and bundles with AI risk scanning
API & MCP surface
REST API — 175+ endpoints under /api/v1/ with Sanctum auth, cursor pagination, auto-generated OpenAPI 3.1 at /docs/api
MCP Server — 670+ Model Context Protocol tools across 60+ domains (stdio + HTTP/SSE + OAuth2/PKCE)
Real-World Action governance — ActionProposal flow gates assistant tool calls, integration writes, and git pushes through a per-tier risk policy with auto-execute on approval
Public discovery endpoint — GET /.well-known/fleetq returns a config-gated capability manifest so external AI tools can auto-configure
Live team graph — /team-graph page with real-time updates via Laravel Reverb WebSockets
Structured MCP errors — canonical gRPC-style error codes (UNAVAILABLE, PERMISSION_DENIED, RESOURCE_EXHAUSTED, DEADLINE_EXCEEDED, INVALID_ARGUMENT, FAILED_PRECONDITION, NOT_FOUND, INTERNAL) with retryable hints — agents know when to retry vs. fail fast
Per-tool deadlines — optional deadline_ms parameter on every MCP tool; agents can bound wall-clock time per call
OpenTelemetry tracing — OTLP HTTP exporter, Jaeger all-in-one via docker compose --profile observability up, spans for MCP tool → AI gateway → LLM provider
MCP client compatibility — Claude Desktop, Claude.ai, ChatGPT Apps, Cursor, Codex, Claude Code, Gemini CLI, any OAuth2 client
Infrastructure
Queue Management — Laravel Horizon with 6 priority queues and auto-scaling
Testing — regression test suites for agent outputs with automated evaluation
Per-Call Working Directory — local/bridge agents can operate in a configured working directory per-agent, isolated project contexts
Use Cases
FleetQ is built for teams running AI agents in production, not toy demos.
Autonomous dev pipelines — agent opens PR → CI runs → reviewer agent approves → merge → deploy. Human approves only on risk signals.
Customer support triage — bug report widget → agent extracts reproduction steps from console/network log → experiment runs → notifies reporter with fix or agent-generated workaround.
Multi-agent research — crew of Strategist + Researcher + Writer with QA reviewer. Each step weighted by domain rubric.
Scheduled content ops — continuous project runs daily, each run executes a DAG: draft → review → SEO-check → publish → schedule social.
Incident response — PagerDuty/Sentry signal → trigger rule → diagnosis agent → human approval on runbook action → Slack notify.
GPU workloads — agent calls gpu_compute skill on RunPod serverless (Whisper, FLUX, Bark) as part of a larger workflow, with cost accounting.
Local-first agent dev — Ollama + Codex + Claude Code auto-detected, zero API cost for prototyping; switch to cloud providers for production.
Bring FleetQ into Claude — expose your internal data + tools as MCP server, Claude Desktop/ChatGPT/Cursor can drive the platform programmatically.
How FleetQ compares
FleetQ
n8n
CrewAI
LangGraph
Make.com
Open source
✅ AGPLv3
✅ Sustainable Use
✅ MIT
✅ MIT
❌ Proprietary
Visual DAG builder
✅ 8 node types
✅ (not AI-first)
❌
❌
✅
Multi-agent crews
✅ 7 process types
❌
✅
✅ (build-your-own)
❌
MCP server (native)
✅ 670+ tools
❌
❌
❌
❌
Human-in-the-loop
✅ native
⚠️ workaround
⚠️ code
⚠️ code
⚠️ approve-node
Budget ledger + locks
✅ pessimistic
❌
❌
❌
❌
Audit trail
✅ every action
✅
❌
❌
✅
BYOK + local LLMs
✅ both
⚠️ BYOK only
⚠️ depends
⚠️ BYOK
❌
Self-hosted
✅ Docker Compose
✅
n/a (library)
n/a (library)
❌
Agent evolution (self-improve)
✅
❌
❌
❌
❌
OpenTelemetry tracing
✅ native
❌
❌
⚠️ partial
❌
Credit/usage metering
✅ per-team/project
❌
❌
❌
per-workspace
TL;DR — if you're building production agent systems with LLMs and want visual workflows + MCP + human oversight, FleetQ is the only platform that bundles all of it.
Quick Start (Docker)
git clone https://github.com/escapeboy/agent-fleet-o.git
cd agent-fleet
make install
This will:
Copy .env.example to .env
Build and start all Docker services
Run the interactive setup wizard (database, admin account, LLM provider)
Then open http://localhost:8000 in your browser. The setup page will guide you through creating your admin account.
Alternative: Run php artisan app:install for an interactive CLI setup wizard that also seeds default agents and skills.
Authentication
No email verification — the self-hosted edition skips email verification entirely. Accounts are active immediately on registration.
Single user — all registered users join the default workspace automatically.
No-Password Mode (local installs)
If you're running FleetQ locally on your own machine and don't want to enter a password on every visit, set APP_AUTH_BYPASS=true in .env:
APP_AUTH_BYPASS=true # Auto-login as first user
APP_ENV=local # Required — bypass is disabled in production
With bypass enabled, the app logs you in automatically on every request. A logout link is still shown but you'll be logged back in on the next page load — this is intentional.
Warning: Never set APP_AUTH_BYPASS=true on a server accessible from the internet.
Configuration
All configuration is in .env. Key variables:
# Database (PostgreSQL required)
DB_CONNECTION=pgsql
DB_HOST=postgres
DB_DATABASE=agent_fleet
# Redis (queues, cache, sessions, locks)
REDIS_HOST=redis
REDIS_DB=0 # Queues
REDIS_CACHE_DB=1 # Cache
REDIS_LOCK_DB=2 # Locks
# LLM Providers -- at least one required for AI features
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GOOGLE_AI_API_KEY=
# Auth bypass -- local no-password mode (never use in production)
APP_AUTH_BYPASS=false
Additional LLM keys can be configured in Settings > AI Provider Keys after login.
To use local models (Ollama, LM Studio, vLLM):
LOCAL_LLM_ENABLED=true
LOCAL_LLM_SSRF_PROTECTION=false # set false if Ollama is on a LAN IP (192.168.x.x)
LOCAL_LLM_TIMEOUT=180
Then configure endpoints in Settings > Local LLM Endpoints.
SSH Host Access
Agents can execute commands on the host machine (or any remote server) via SSH using the built-in SSH tool type. This is useful for running local scripts, interacting with the filesystem, or orchestrating host-level processes from an agent.
How it works
The platform stores SSH private keys encrypted in the Credential vault.
An SSH Tool is configured with host, port, username, credential_id, and an optional allowed_commands whitelist.
On the first connection to a host, the server's public key fingerprint is stored via TOFU (Trust On First Use). Subsequent connections verify the fingerprint — a mismatch raises an error to prevent MITM attacks.
Manage trusted fingerprints via Settings > SSH Fingerprints or the tool_ssh_fingerprints MCP tool.
Setup (Docker — connecting container to host)
The containers reach the host machine via host.docker.internal, which is pre-configured in docker-compose.yml via extra_hosts: host.docker.internal:host-gateway.
Step 1 — Enable SSH on the host
OS
Command
macOS
System Settings → General → Sharing → Remote Login → On
The platform is a single Laravel 13 monolith that exposes three coequal control surfaces over the same domain layer: the Livewire admin UI, a Sanctum-authenticated REST API at /api/v1/* (~175 endpoints), and AgentFleetServer — an MCP server with 670+ tools across 60+ domains served over both HTTP/SSE and local stdio. Inbound signals (webhook, RSS, IMAP, Slack, Telegram, and the rest of the 20+ connectors) flow through IngestSignalAction and the TriggerRule evaluator into the domain layer, where the ExperimentStateMachine walks a 20-state pipeline by emitting ExperimentTransitioned events whose listeners dispatch the next BaseStageJob onto Horizon-managed Redis queues. Stage jobs talk to LLMs through the PrismPHP-backed AI Gateway (rate-limit, budget, idempotency, semantic-cache, schema-validation, usage-tracking middleware + circuit breakers + provider fallbacks), invoke Tool instances translated to PrismPHP tool calls (MCP stdio/HTTP, built-in bash/filesystem/browser, SSH with TOFU fingerprints), park ApprovalRequest/HumanTask records for the human-in-the-loop inbox, and persist Artifact versions plus deliver outbound messages over Email/Telegram/Slack/Webhook/ntfy. State and tenant data live in Postgres 17 with pgvector (semantic cache, UUIDv7 primary keys, JSONB+GIN indexes); Redis 7 carries the six Horizon queues, application cache, and pessimistic budget locks. Optional Docker Compose profiles add Reverb for the live team-graph WebSocket, browserless for browser tools, searxng for web search, a voice worker (LiveKit/Deepgram), a sandboxed bash sidecar, the fleetq-bridge relay, and Jaeger for OpenTelemetry tracing via --profile observability.
Built with Laravel 13, Livewire 4, and Tailwind CSS. Domain-driven design with 45 bounded contexts — table below shows the 17 primary domains:
Domain
Purpose
Agent
AI agent configs, execution, personality, evolution
Crew
Multi-agent teams with lead/member roles
Experiment
Pipeline, state machine, playbooks
Signal
Inbound data ingestion
Outbound
Multi-channel delivery
Approval
Human-in-the-loop reviews and human tasks
Budget
Credit ledger, cost enforcement
Metrics
Measurement, revenue attribution
Audit
Activity logging
Skill
Reusable AI skill definitions
Tool
MCP servers, built-in tools, risk classification
Credential
Encrypted external service credentials
Workflow
Visual DAG builder, graph executor
Project
Continuous/one-shot projects, scheduling
Assistant
Context-aware AI chat with 28 tools
Marketplace
Skill/agent/workflow sharing
Integration
External service connectors (GitHub, Slack, Notion, Airtable, Linear, Stripe, Generic)
Docker Services
Service
Purpose
Port
app
PHP 8.4-fpm
--
nginx
Web server
8080
postgres
PostgreSQL 17
5432
redis
Cache/Queue/Sessions
6379
horizon
Queue workers
--
scheduler
Cron jobs
--
vite
Frontend dev server
5173
Common Commands
make start # Start services
make stop # Stop services
make logs # Tail logs
make update # Pull latest + migrate
make test # Run tests
make shell # Open app container shell
Or with Docker Compose directly:
docker compose exec app php artisan tinker # REPL
docker compose exec app php artisan test # Run tests
docker compose exec app php artisan migrate # Run migrations
Decision-Model Eval Harness
jev:eval scores a decision model — TypeSafe's Jev, a second System One
endpoint, or an ordinary chat LLM — against a JSONL dataset, and jev:report
turns the recorded answers into accuracy, calibration, coverage, latency, cost
and determinism numbers. Nothing in the request path calls it: it is a
measurement tool, not a production dependency.
The API key never touches the repo
TYPESAFE_API_KEY lives in 1Password and reaches the process only through
op run. .env.op (committed) holds the reference, never the value:
TYPESAFE_API_KEY="op://AI Agent/Jev API Key/credential"
The baseline drivers follow the same rule — .env.op also carries references
for ANTHROPIC_API_KEY, GEMINI_API_KEY / GOOGLE_AI_API_KEY and
OPENAI_API_KEY, and each command passes only the names it needs through
docker compose run -e.
Every command that calls the API is run through op run, which resolves the
reference into the child process and masks it in the output. Do not pass
--no-masking, do not op read the item into a variable, and do not write the
value into .env, a compose file or a fixture.
Running an eval end to end
# 1. Build the datasets.
python3 base/scripts/jev-eval/export_next_tool_dataset.py # -> next-tool.jsonl
python3 base/scripts/jev-eval/export_routing_v2_dataset.py # -> routing-v2.jsonl + routing-v2-synth.jsonl
python3 base/scripts/jev-eval/export_domain_prefilter_dataset.py # -> domain-prefilter.jsonl
# both write into ~/jev-eval/datasets/fleetq/
# 2. Make them reachable from the container (storage/ is bind-mounted).
mkdir -p storage/app/jev-eval
cp ~/jev-eval/datasets/fleetq/*.jsonl storage/app/jev-eval/
# 3. Run the eval. The key is injected by op, by name, for this process only.
export OP_SERVICE_ACCOUNT_TOKEN=$(cat ~/.config/op/sa-token)
op run --env-file=.env.op -- docker compose run --rm -e TYPESAFE_API_KEY app \
php artisan jev:eval storage/app/jev-eval/routing-v2.jsonl --driver=jev --split=test --concurrency=8
# 4. Report on the run id the eval printed.
docker compose exec app php artisan jev:report <run-id> \
--dataset-path=storage/app/jev-eval/routing-v2.jsonl
Measuring determinism — the same requests sent N times:
op run --env-file=.env.op -- docker compose run --rm -e TYPESAFE_API_KEY app \
php artisan jev:eval storage/app/jev-eval/routing-v2.jsonl --driver=jev --split=dev --repeat=3
Comparing Jev against a chat model on the same dataset — the LLM drivers use the
platform AI gateway, so they need no TypeSafe key:
op run --env-file=.env.op -- docker compose run --rm -e ANTHROPIC_API_KEY app \
php artisan jev:eval storage/app/jev-eval/routing-v2.jsonl --driver=haiku --split=test --team=<team-id>
--team is required: the gateway logs every call to the tenant-scoped
llm_request_logs, and an eval has no tenant of its own. The gateway also caps
Anthropic at 60 requests per minute across the whole process, so the LLM drivers
publish their own requests_per_minute and the eval throttles itself to it —
two of these running flat out in parallel will still starve each other.
gemini-flash and gpt-mini exist for one reason: on a dataset whose gold
labels were produced by Claude models, an Anthropic baseline is scoring its own
homework. A model family that had no hand in the labels is the only baseline
that means anything there.
When the metered Anthropic key is unavailable, claude_cli_haiku and
claude_cli_sonnet run the same prompts through the local Claude Code CLI on
its subscription credentials. They need no key, no --team and no gateway:
op run --env-file=.env.op -- php artisan jev:eval ~/jev-eval/datasets/fleetq/routing-v2.jsonl --driver=jev --split=test
op run --env-file=.env.op -- php artisan jev:report
Options
jev:eval:
Option
Default
Meaning
--driver
jev
A key from config/decision.php: jev, jeff, haiku, sonnet, gemini-flash, gpt-mini, claude_cli_haiku, claude_cli_sonnet
--split
test
dev, test, or all. 20/80, decided by a hash of the case id
--repeat
1
Send each case N times; feeds the determinism columns
--concurrency
8
Cases in flight at once, for drivers that support batching
--limit
0
Stop after N cases, taken in file order so two drivers stay comparable
--team
—
Team the LLM drivers log their gateway calls under (required for haiku/sonnet)
jev:report:
Option
Meaning
run_id...
One or more runs; several put different drivers in one table set
--group-by=meta.source
Split every table by a dataset meta key
--where=meta.source=assistant_turn
Keep only cases matching a meta key (comma-separated values allowed)
--questions=domain
Report only these question ids
--multi-label
Score each case as one label SET (Noul per label) instead of per question
--dataset-path=
The dataset file; required by --group-by, --where and the variant analysis
Accuracy is printed with a 95% Wilson interval, which is what makes a
per-source table readable when some sources have only a handful of cases.
Each question type also gets its own block under the headline table:
Type
Extra metrics
Choice
top-2 accuracy (gold within the two highest-probability options), per-class precision / recall / F1, confusion pairs sorted by count
Noul
precision, recall and F1 of the positive class at t=0.5, PR-AUC (average precision), and the gold positive rate printed next to accuracy
Score
MAE on the level index, and binary accuracy for level 0 vs level > 0
The run stays under Jev's published ceilings (1,200 requests/minute and 250,000
tokens/second) on its own. A case whose state plus longest question is estimated
over 32k tokens is rejected and logged by id, never truncated — a shortened
state is a different case, and scoring it would move the accuracy number without
saying so.
the registry domain of the tool a description came from
domain-prefilter.jsonl
which domains can be ruled out (multi-label)
every MCP domain the assistant drew a tool from in that turn
next-tool.jsonl is one case per tool call an agent actually made inside a
session belonging to an experiment that reached completed. The state carries
the task brief and the steps already taken; the gold answer is the tool that was
in fact chosen next. The assistant's own narration is deliberately excluded — it
routinely names the next tool, which would turn routing into string extraction.
This is agent behaviour, not FleetQ routing.
routing-v2.jsonl never scores against what an agent decided. Each gold answer
is a configuration fact or an explicit human choice already in the database:
a user asked the assistant for something and it called tools from exactly one
MCP domain (turns spanning two domains are dropped, because the gold would be
ambiguous); a signal was routed into an experiment whose workflow template was
configured; a human created an experiment and assigned a specific agent to it.
meta.source records which, so a subset can be scored on its own. A single
source task contributes at most 15 cases.
routing-v2-synth.jsonl is generated from the tool registry's own descriptions
to cover the domains production data never exercises. Every case carries
meta.source = "synthetic" and it lives in its own file so it is never mixed
into headline numbers — score it separately or not at all.
domain-prefilter.jsonl asks a different question from routing-v2: not "which
one domain handles this" but "which domains can be ruled out" — the shape a
prefilter in front of a 700-tool MCP server needs. One Noul per domain, all 67
in a single request, gold true for every domain the turn drew a tool from.
Multi-domain turns are kept here, because a turn spanning two domains is a
correct multi-label answer rather than an ambiguous one. Score it with
jev:report --multi-label, which sweeps the threshold and reports the tightest
prefilter that still keeps essentially every true label.
On routing-v2 headline numbers: only the assistant_turn source belongs in
one. The signal_workflow and experiment_agent sources encode a
configuration fact that the request text cannot support — a Sentry bug report
reads like a signal whatever the tenant configured it to trigger — and
model_tier has a single value across every case where prod data can derive it.
Report those per source, never merged:
The domain option list and its one-sentence descriptions are read out of the
registry by base/scripts/jev-eval/mcp_domain_registry.py; add a tool group and
the option appears on its own.
Upgrading
make update
This pulls the latest code, rebuilds containers, runs migrations, and clears caches.
TL;DR of AGPLv3: You can self-host, modify, and run FleetQ for free — including commercial use. If you offer FleetQ as a hosted service to others, you must open-source your modifications. Questions? See our AGPLv3 FAQ.
A
Agent Fleet O
escapeboy·81 Sterne·
Open-source AI agent orchestration platform — self-hosted mission control for autonomous multi-agent systems. Visual DAG workflows, 670+ MCP tools, human-in-the-loop approvals. Works with Claude, GPT-
Open-source AI agent orchestration platform — self-hosted mission control for autonomous multi-agent systems. Visual DAG workflows, 670+ MCP tools, human-in-the-loop approvals. Works with Claude, GPT-4o, Gemini, Ollama, Codex.
Tools
Tools
Noch keine Tool-Liste. Remote-Server erhalten eine nach dem ersten Health-Check.
FAQ
Fragen
Wie installiere ich Agent Fleet O?
Kopieren Sie auf dieser Seite die stdio-Konfiguration für Cursor, Claude Desktop, Claude Code oder VS Code. Sie nutzt die Installation des Herausgebers, keinen PlugMCP-Proxy.