Dunetrace

Runtime reliability for AI agents. Structural and semantic detection, runtime prevention, native root cause, and one-click fixes.

Star us ⭐
If Dunetrace helps you, consider giving it a ⭐ on top right, it helps others find the project.
The problem
AI agents fail silently:
- ✓ API returns 200 ✓ Latency is normal ✓ Cost looks normal
- ✗ The upstream API returned an error body. The agent invented the numbers and reported success.
- ✗ Two agents delegated in a circle. Eight runs, all green, no progress.
- ✗ A document your agent read last week wrote an instruction into its memory. It fired today.
Tracers answer "what happened?" i.e. after you already know it broke. Dunetrace answers "is something breaking right now?" with deterministic, zero-LLM checks on every run, and in the request path, where a policy can block the action before it executes.
Five pillars, one platform
Dunetrace covers the full agent reliability lifecycle, not just one slice of it:
Quick Start
1. Start the backend
git clone https://github.com/dunetrace/dunetrace
cd dunetrace && cp .env.example .env
docker compose -f docker-compose.ghcr.yml up -d # localhost, auth off — see docs/operations.md to deploy
curl -s localhost:8001/ready && curl -s localhost:8002/ready # both {"status":"ok",...}
2. Install the SDK
pip install dunetrace # Python
npm install dunetrace # Node.js / TypeScript
# Or skip steps 2-3 entirely: the skill installs the SDK and instruments your
# code. Run /dunetrace-setup (Claude Code, Cursor) or $dunetrace-setup (Codex).
npx skills add dunetrace/dunetrace-skills --skill dunetrace-setup
3. Instrument your agent
Python
from dunetrace import Dunetrace
import openai
dt = Dunetrace()
dt.init(agent_id="support-agent") # auto-instruments installed clients (OpenAI, Anthropic, Mistral, Bedrock, LangChain, CrewAI, httpx, requests)
@dt.agent("support-agent", model="gpt-4o")
def my_agent(question: str) -> str:
resp = openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": question}],
)
return resp.choices[0].message.content # LLM + tool calls tracked automatically, no manual hooks
TypeScript / Node.js
import { Dunetrace, autoInstrument } from "dunetrace";
import OpenAI from "openai";
const dt = new Dunetrace();
autoInstrument({ openai: OpenAI }); // patches OpenAI + outbound fetch; add `anthropic:` / `mistral:` too, or wrap one client with dt.wrapOpenAI()
const openai = new OpenAI(); // constructed after the patch — still tracked
await dt.run("support-agent", { model: "gpt-4o" }, async (run) => {
await openai.chat.completions.create({ model: "gpt-4o", messages });
run.finalAnswer(); // LLM + tool calls tracked automatically, streaming included
});
→ TypeScript auto-instrumentation
Try the built-in failure scenarios
cd packages/sdk-py # Python
python examples/basic_agent.py # No LLM calls, no API key
SCENARIO=tool_loop python examples/langchain_agent.py # TOOL_LOOP via LangChain — needs OPENAI_API_KEY
SCENARIO=failures python examples/decorator_agent.py # TOOL_LOOP, RETRY_STORM, RAG_EMPTY_RETRIEVAL
cd ../sdk-ts && npm install # TypeScript — needs Node 22+
ollama pull llama3.2 # and Ollama, so no API key is needed
npm run example:vercel-ai # Happy path
npm run example:vercel-ai:loop # TOOL_LOOP → detect → explain, end to end
More, including approvals, policies, agent memory and voice → examples index
Open the dashboard: http://localhost:3000
Detectors
34 detectors run on every completed run — no configuration, no LLM. A few of the main ones:
Each alert carries what fired, why it matters, a concrete fix, and rate context (first occurrence / recurring / systemic). → docs/detectors.md for all 34
- Multi-agent — nested
dt.run()calls auto-link into a delegation graph thatDELEGATION_LOOPandHANDOFF_CONTEXT_LOSSread → docs/multi-agent.md - Agent memory — instrument memory writes/reads and
MEMORY_POISONINGflags adversarial content persisted into them → docs/memory.md - Custom detectors — describe one in plain English; it runs in shadow mode until you approve the fire rate → docs/detectors.md
- Detector packs — opt-in bundles per org; the voice pack adds 9 detectors → detector packs · voice pack
Semantic evaluation
Opt-in LLM judgment for what structural checks can't see — seven DeepEval-backed evaluators (hallucination, task completion, task-understanding failure, off-topic drift, user frustration, confusion loops, sycophancy). Post-hoc and sampling-based, never in your agent's request path. Its own container, off by default:
SEMANTIC_WORKER_ENABLED=true
Dashboard

Live at http://localhost:3000. Auto-refreshes every 15s.
Alerts
Slack and generic webhook (PagerDuty, Linear, custom).
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
SLACK_MIN_SEVERITY=LOW # LOW | MEDIUM | HIGH | CRITICAL
DIGEST_ENABLED=true # weekly digest of top failure types, Monday 9am UTC
Diagnose & fix
Click Explain + on any alert: Dunetrace analyzes the run's own stored events (no third-party tracer) and returns a cause plus a fix — either a runtime policy it applies itself (tool loops, retry storms, runaway step counts) or a prompt/code diff you copy in or open as a draft PR. Fix effectiveness is tracked automatically.
Policies
Runtime guardrails that fire mid-run — before a failure propagates. Defined in code or in the dashboard (the SDK refetches every 60s).
dt.add_policy( # stop, switch_model, escalate, inject, ...
name="cap tool calls",
condition={"trigger": "tool_call_count", "operator": "gt", "value": 5},
action={"type": "stop"},
)
dt.add_policy( # human-in-the-loop: blocks the call until
name="approve-wires", # someone approves in Slack or the dashboard,
condition={"trigger": "before_tool_call", "operator": "eq", "value": "wire_money"},
action={"type": "require_approval", "params": {"timeout_s": 300}}, # fail-closed on timeout
)
→ docs/policies.md · docs/approvals.md
MCP server
Query agent signals from Claude Code, Cursor, or Codex — "what failed in the last 24 hours?" — without leaving your editor.
pip install dunetrace-mcp
31 tools covering agents, runs, signals, fixes, issues, policies, custom detectors and voice calls. Claude Code registers the server automatically in ~/.claude.json (restart to load); Cursor and Codex need one config block.
Architecture
Services share one Postgres and no message broker: the SDK ships events to the ingest API, and each worker polls for what it needs next. The core pipeline runs with no LLM key; three workers are opt-in.
→ docs/architecture.md — services, data flow, detection paths, schema · docs/operations.md — deploying, probes and metrics, retention, redaction, rate limiting
Integrations
Model providers — OpenAI, Anthropic, Mistral and AWS Bedrock, auto-instrumented with no call-site changes.
- Auto-instrumentation (what's patched, streaming, agent_id resolution)
- Mistral (hyperscaler-hosted clients, EU-resident evaluation)
Evaluation & tracing
- OpenTelemetry export (Datadog, Grafana, Honeycomb, Signoz, any OTLP backend)
- OpenTelemetry ingestion (send your existing OTel traces to Dunetrace: OpenLIT, Traceloop, OTel contrib)
- External evaluation (Langfuse, LangSmith, Braintrust, generic push)
- Semantic evaluation (Dunetrace's own DeepEval-backed layer)
Fix & workflow
- GitHub App (automated draft PRs)
- Slack & Linear alerts
- Coding agents — Claude Code, Cursor, Codex (MCP)
Voice
- ElevenLabs (correlate TTS cost and voice choices with agent behavior)
- Wiring a voice framework
- Voice metrics (call-level view of a voice agent)
- Voice detector pack (9 detectors)
- Custom Python agent
- LangChain / LangGraph
- CrewAI
- Pydantic AI
- AutoGen (Microsoft)
- OpenAI Agents SDK
- Haystack 2.x
- Hermes Agent (Nous Research)
- LlamaIndex
- smolagents (Hugging Face)
- TypeScript / JavaScript
- Langdock
- Dify
- LiteLLM
- Vercel AI SDK
- Policies
- Human-in-the-loop approvals
- Agent state machine
Contributing
Fork, branch, change, make test, PR — open an issue first for new integrations or architecture changes. Requires Python 3.11+, Node.js 22+, Docker + Docker Compose.
→ CONTRIBUTING.md (setup and workflow) · good first issues · adding a detector
Contact
Dunetrace UG (haftungsbeschränkt) · Kolonnenstr. 8, 10827 Berlin, Germany · [email protected]