An open-source testing and observability framework for multi-agent delegation, tool execution, MCP workflows, permissions, sandboxed actions, prompts, and runtime behavior.
Status: Experimental — v0.1.0 release candidate. Suitable for local development, evaluation, and policy testing; not production-hardened infrastructure.
Testing and observability for multi-agent delegation, MCP tools, permissions, sandboxed actions, prompts, and workflow replay.
2026-10-27
Estrelas
86
Ferramentas
—
Disponibilidade
—
Why this exists
Most model evaluation stops at input → model → output. Multi-agent systems add behavior that a final answer cannot explain:
Which agent delegated a task, to whom, and why?
What context, permissions, tools, MCP servers, and budget did the child receive?
Did an agent attempt privilege escalation, repeat work, or enter a loop?
Was the delegation efficient, and was the child result actually integrated?
Can the run be inspected, compared, or replayed without repeating side effects?
MAWL makes those decisions explicit, policy-controlled, and traceable. Prompts are versioned assets; delegation is a first-class runtime event; deterministic rules evaluate behavior independently from optional model judges.
Key capabilities
Area
What is implemented
Agent runtime
Typed model actions, task limits, cancellation, retries, and deterministic mock execution
Delegation
Parent/child task graph, target and capability checks, depth/fan-out limits, loop detection, and result integration events
Tool execution
Registry-based invocation with schema validation, allowlists, permissions, policy, optional approval, redaction, timeout, and audit
MCP
Official stdio and Streamable HTTP client adapters plus an in-memory mock connector; MCP data remains untrusted
Sandbox
Restricted local-process adapter and optional Docker provider interface with auditable lifecycle events
Identity and permissions
Runtime-issued agent identity, local development auth, deny-by-default permission and contextual policy engines
The actual CLI output is JSON and includes the run ID, every task/agent/status, all eight evaluation dimensions, derived metrics, and the JSONL trace path.
Delegation observability
A trace records behavior rather than only the final answer:
Evaluation scores are normalized diagnostics, not mathematical truth or a security guarantee.
Example workflows
Workflow
Purpose
Expected result
01-basic-delegation.yaml
Orchestrator delegates one bounded research task
Completes
02-parallel-research.yaml
Two research tasks fan out and an analyst joins them
Completes
03-review-chain.yaml
Researcher → analyst → reviewer → evaluator
Completes
04-permission-denied.yaml
Direct delegation to a non-allowlisted target
Fails safely
05-mcp-untrusted-output.yaml
Hostile MCP content is treated as data in the test harness
Completes safely
06-delegation-loop.yaml
Agent A → B → A recursion
Fails safely
07-human-approval.yaml
Approval-provider checkpoint behavior
Completes; approval logic is tested separately
08-budget-exceeded.yaml
Workflow-wide token budget exhaustion
Fails safely
09-bad-vs-good-orchestrator.yaml
Good baseline for comparative evaluation
Completes
Run any positive example with:
pnpm mawl run workflows/02-parallel-research.yaml
The negative examples intentionally return a non-zero exit code when the requested action is blocked.
Good vs. bad orchestrator
The lab compares a bounded, least-privilege orchestration trace with a synthetic bad trace containing duplicate work, leaked context, escalated permission, and an unauthorized tool request. RunComparator and DelegationEvaluator report changes in:
tool-rerun blocks tool names or manifests classified as external side effects unless the embedding application supplies explicit permission for that exact tool. The CLI supplies no side-effect permissions.
Prompts are first-class assets
This is not a directory of free-form prompt snippets. YAML assets under prompts/ include:
stable ID and semantic version;
owner, type, purpose, and known risks;
declared input schema and expected output;
allowed actions and recommended capabilities;
content hash verified by PromptRegistry;
provenance and trust classification when assembled at runtime.
Runtime and security policy layers remain separate from untrusted user, tool, MCP, and child-agent content. Prompt quality tests detect duplicate IDs, invalid versions, missing metadata, variable/placeholder errors, unexpected hashes, and undeclared references.
Security model
Permissions and contextual policies deny unmatched requests by default.
Agent execution identities are issued by the runtime and linked to workflow/task/session IDs.
Child context, authority, and budget are reduced from the parent.
MCP responses and model/tool/child output are untrusted data, not authority.
Secret references and output redaction limit accidental exposure.
Delegation depth, ancestry, task count, retries, calls, tokens, output, and runtime are bounded.
Replay refuses external side effects without an explicit per-tool permission.
The restricted local sandbox is intended for development and policy testing. It is not equivalent to hardened container, VM, microVM, or kernel isolation, and network denial is best-effort. MAWL does not claim to be prompt-injection-proof, sandbox-escape-proof, or fully zero-trust. Read the threat model and security policy.
Using a real model provider
The default path always uses MockModelProvider. An optional OpenAI-compatible chat-completions adapter is available in @mawl/providers:
cp .env.example .env
Populate MODEL_PROVIDER, MODEL_ENDPOINT, and MODEL_API_KEY from a secret manager, then inject OpenAICompatibleProvider into the runtime. The CLI intentionally does not auto-load credentials. See provider integration.
All default tests use local mock providers and require no external API access:
pnpm install --frozen-lockfile
pnpm typecheck
pnpm lint
pnpm build
pnpm test
pnpm test:e2e
pnpm test:security
pnpm demo
pnpm mawl doctor
To extend the framework, add YAML agents under agents/, versioned prompts under prompts/, workflows under workflows/, tool manifests through ToolRegistry, MCP adapters through McpConnection/McpConnector, rules through RuleEvaluator, and scenario coverage under tests/ or scenarios/. See the documentation index and contributing guide.
Roadmap
distributed execution and atomic shared budgets;
hardened remote container or microVM sandbox providers;
OpenTelemetry exporters and remote observability storage;
additional model, identity, and MCP adapters;
workflow visualization UI and benchmark datasets.
Contributing, security, and license
Contributions to agents, prompts, tools, MCP connectors, workflow scenarios, evaluators, and security tests are welcome. Read CONTRIBUTING.md and CODE_OF_CONDUCT.md.
Report vulnerabilities privately as described in SECURITY.md. Do not place sensitive vulnerability details in public issues.
Ainda não há lista de ferramentas. Servidores remotos recebem uma após a primeira verificação de saúde.
Perguntas frequentes
Perguntas
Como instalo Multi Agent Workflow Lab?
Copie nesta página a configuração stdio para Cursor, Claude Desktop, Claude Code ou VS Code. Ela usa a instalação do publicador, não um proxy do PlugMCP.
Multi Agent Workflow Lab precisa de uma chave de API?
Este registro não lista nenhuma variável de ambiente obrigatória.
Qual transporte Multi Agent Workflow Lab usa?
Multi Agent Workflow Lab roda localmente via stdio.
Quando Multi Agent Workflow Lab foi verificado pela última vez?
Última verificação: 2026-10-27. Testing and observability for multi-agent delegation, MCP tools, permissions, sandboxed actions, prompts, and workflow replay.