Proxymic ("DataDog" for AI Agents)
Proxymic is a transparent MCP gateway that makes agent-to-server, agent-to-tool, and client-to-server communication effortless — while caching, securing, and giving you full observability into every tool call your AI agents make. Slash API costs by 40-70%. Kill agentic loops before they burn tokens. Detect anomalies, hallucinations, and prompt injections in real time. All without modifying a single line of agent or server code.
Drop it between any agent (Claude, Cursor, custom) and any MCP server. It just works.
Why
Every AI agent today talks to tools — file systems, databases, APIs, browsers, terminals. But the protocol between agents and these tools (MCP over stdio) is brittle:
- Identical prompts cost the same every time. An agent asking "list files in /src" pays full LLM cost on every single call — even when the directory hasn't changed.
- No built-in security. Any tool the server exposes, the agent can call — including
exec_command,bash,write_file. One prompt injection and your filesystem is gone. - Agents get stuck in loops. A misconfigured agent will hammer the same tool thousands of times before anyone notices.
- A single hung tool call takes down the whole agent. No timeouts. No circuit breakers. No recovery.
These aren't hypotheticals. If you've built with AI agents, you've hit every one of these.
Background
I follow AI infrastructure closely. Two of my friends were building AI startups, and their biggest pain point was the same: LLM API bills were eating them alive. Every repeated prompt cost money. Every agent loop burned tokens. I read about startups hitting their yearly API budget in a month. Microsoft capped employee token usage. Salesforce had the same problem.
So I built an AI Gateway — semantic caching, multi-strategy matching, adaptive thresholds. It saved 40-70% on LLM costs. But the deeper I went, the more I realized: the real problem isn't just API costs. It's the entire MCP layer. AI agents call tools hundreds of times a day. Each call is slow, unsecured, uncached. MCP has no Kong, no Cloudflare, no API Gateway. So I built one.
Proxymic is that layer. One binary. Zero dependencies. It caches, secures, and hardens every tool call between your AI agent and its MCP servers. Same way Kong became essential for microservices — Proxymic becomes essential for AI agents.
What Cloudflare and Datadog did for web infrastructure, Proxymic does for AI agent infrastructure.
Try It Now
No signup. No install. It's a live demo with an interactive "Try It" console — type a prompt and see exactly how caching, rate limiting, and the agent health system respond in real time.
The demo page includes:
- An interactive tool call console
- Live Agent Health dashboard showing every tool call, anomaly detection, and savings estimates
- Cache hit/miss indicators
- Real-time metrics and agent behavior analysis
How It Works
Proxymic intercepts every JSON-RPC message flowing between an agent and its MCP server. It passes through initialization and discovery transparently, but intercepts every tools/call request to apply seven layers of hardening:
Agent (Claude, Cursor, custom)
│ stdin/stdout (JSON-RPC over MCP)
▼
┌──────────────────────────────────────────────────┐
│ Proxymic Gateway │
│ │
│ ┌──────────── Request Pipeline ────────────┐ │
│ │ │ │
│ │ 1. Security Blocklist │ │
│ │ ├── Blocked? → Error │ │
│ │ └── Allowed? → Continue │ │
│ │ │ │
│ │ 2. Rate Limiter │ │
│ │ ├── Exceeded? → Error │ │
│ │ └── OK? → Continue │ │
│ │ │ │
│ │ 3. Circuit Breaker │ │
│ │ ├── Open? → Error │ │
│ │ └── Closed? → Continue │ │
│ │ │ │
│ │ 4. Cache Lookup (multi-strategy) │ │
│ │ ├── Semantic HIT → <3ms return │ │
│ │ ├── Exact HIT → <1ms return │ │
│ │ └── MISS → Forward │ │
│ │ │ │
│ │ 5. Response Deduplication │ │
│ │ ├── 3rd+ identical → Placeholder │ │
│ │ └── Unique → Store + pass │ │
│ │ │ │
│ │ 6. Timeout Guard + Auto-Restart │ │
│ │ ├── OK → Cache + return │ │
│ │ └── Timeout → Kill + restart │ │
│ │ │ │
│ │ 7. Agent Health (observability) │ │
│ │ └── Every call logged, analyzed, │ │
│ │ displayed in real-time │ │
│ └────────────────────────────────────────────┘ │
│ │
│ ┌───────── Background Systems ──────────┐ │
│ │ │ │
│ │ • Health checks every 15s │ │
│ │ • Process monitoring + auto-restart │ │
│ │ • Agentic loop detection & kills │ │
│ │ • Metrics server (:9090/metrics) │ │
│ │ • Structured JSON logging │ │
│ └────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────┘
│ stdin/stdout (JSON-RPC over MCP)
▼
MCP Server (filesystem, database, browser, terminal, ...)
Core Capabilities
Smart Caching
Not just exact-match. Proxymic uses a cascade of five matching strategies to catch semantically identical prompts even when they differ in phrasing:
The cache self-tunes — adaptive threshold adjustment means it learns what "similar enough" means for your particular workload.
Response Deduplication
When an agent asks the same question 100 times in parallel (they do this), Proxymic forwards only one request to upstream. The other 99 get a placeholder response. Saves context window space and eliminates redundant compute.
Agentic Loop Killer
Detects runaway agents before they burn through your token budget. Three stages of escalation:
- Monitor — Track frequency and content similarity per agent session
- Throttle — Insert progressive delays as loop score rises
- Block — Return a clear error with
Retry-Afterheader when the loop is confirmed
The detection uses a combination of request frequency, content similarity, and repetition analysis to distinguish genuine usage from buggy loops.
Circuit Breaker
If the upstream server starts failing, Proxymic opens the circuit — all subsequent calls fail fast without touching the server. After a recovery timeout, it tries a single probe request. If that succeeds, the circuit closes. If not, it waits longer. Prevents cascading failures.
Security Blocklist
Define which tools are allowed and which are blocked at the proxy level. Not at the agent level, not at the server level — at the network boundary. A prompt-injected agent cannot call exec_command or bash even if it tries to.
Agent Health — Live Observability
Every tool call is captured, analyzed, and displayed on a live dashboard:
- Tool call trace — See every call, its arguments, result, and latency
- Anomaly detection — Blocked calls, repeated identical calls, slow responses are flagged automatically
- Savings estimates — See exactly how many tokens and how much latency each cache hit saved
- Performance breakdown — Cache hit rate, circuit breaker state, rate limit hits, all in real time
Quick Start
Option 1: Download Pre-built Binary (fastest)
Grab the binaries from GitHub Releases. You need both the gateway and the demo server (for --demo / --pipe modes):
# Linux
curl -sSL https://github.com/Arnab758/mcp-gateway/releases/latest/download/proxymic-linux-amd64 -o proxymic
curl -sSL https://github.com/Arnab758/mcp-gateway/releases/latest/download/mcp-demo-server-linux-amd64 -o mcp-demo-server
chmod +x proxymic mcp-demo-server
# macOS
curl -sSL https://github.com/Arnab758/mcp-gateway/releases/latest/download/proxymic-darwin-amd64 -o proxymic
curl -sSL https://github.com/Arnab758/mcp-gateway/releases/latest/download/mcp-demo-server-darwin-amd64 -o mcp-demo-server
chmod +x proxymic mcp-demo-server
# Windows (PowerShell)
Invoke-WebRequest -Uri https://github.com/Arnab758/mcp-gateway/releases/latest/download/proxymic-windows-amd64.exe -OutFile proxymic.exe
Invoke-WebRequest -Uri https://github.com/Arnab758/mcp-gateway/releases/latest/download/mcp-demo-server-windows-amd64.exe -OutFile mcp-demo-server.exe
# Run
./proxymic --demo
# → Open http://localhost:9090/ in your browser
Option 2: Clone and Build
# Clone
git clone https://github.com/Arnab758/mcp-gateway.git
cd mcp-gateway
# Build (single static binary, zero runtime dependencies)
go build -o proxymic .
# Build the demo server (for --demo mode)
cd cmd/mcp-demo-server && go build -o ../../mcp-demo-server . && cd ../..
# Run in demo mode
./proxymic --demo
# → Open http://localhost:9090/ in your browser
# → See live traffic, try the interactive playground
Option 3: Docker Compose (recommended)
docker compose up --build
# → Open http://localhost:9090/
Option 4: Docker (manual)
docker build -t proxymic .
docker run -p 9090:9090 proxymic
# → Open http://localhost:9090/
Verify It Works in 2 Minutes
Build everything, then send the same call twice and a dangerous command once. You should see the cache and the security layer kick in:
# 1. Build
cd cmd/mcp-demo-server && go build -o ../../mcp-demo-server . && cd ../..
go build -o proxymic .
go build -o mcp-client ./cmd/mcp-client
# 2. Direct call — hits the server, no gateway
./mcp-client list_files path=/home
# 3. Same call through the gateway — first time is a CACHE MISS (slow, server round trip)
./mcp-client --proxy list_files path=/home
# 4. Same call again through the gateway — CACHE HIT (microseconds, server never called)
./mcp-client --proxy list_files path=/home
# 5. Dangerous command — BLOCKED by the security layer before it reaches the server
./mcp-client --proxy exec_command command="rm -rf /"
If steps 3–4 show a big latency drop (server round trip → microseconds) and step 5 returns a block error, the gateway is working end to end.
Connect Your Agent
Proxymic wraps any MCP server. Point your agent's config at Proxymic instead of the server directly.
Claude Desktop
Edit claude_desktop_config.json and replace your MCP server command with Proxymic:
{
"mcpServers": {
"filesystem": {
"command": "/path/to/proxymic",
"args": ["--config", "/path/to/mcp-gateway.yaml"],
"env": {}
}
}
}
Then create mcp-gateway.yaml with your actual MCP server — here is a working example with the official filesystem server (requires Node):
server:
command: "npx" # Your MCP server command
args: ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/allowed/folder"] # Server args
timeout_seconds: 30
max_restarts: 5
security:
blocked_tools: ["exec_command", "bash", "rm"] # Block dangerous tools at the gateway
No MCP server handy? Point at the bundled demo server instead:
server: command: "./mcp-demo-server" args: [] timeout_seconds: 30 max_restarts: 5
Restart Claude Desktop. Every tool call now passes through Proxymic — cached, secured, observed.
Cursor
In Cursor's MCP configuration, set the command to Proxymic:
{
"mcpServers": {
"my-server": {
"command": "/path/to/proxymic",
"args": ["--config", "/path/to/mcp-gateway.yaml"]
}
}
}
Same mcp-gateway.yaml as above specifies your actual MCP server.
Custom Agent (stdin/stdout)
If your agent communicates with MCP over stdio (most do), just pipe through Proxymic:
# Start Proxymic with your MCP server
./proxymic --config ./mcp-gateway.yaml
Proxymic reads JSON-RPC from stdin, applies the full pipeline (security → rate limit → circuit breaker → cache → dedup), forwards to the server, and writes the response to stdout. Your agent doesn't know Proxymic exists.
Pipe Mode (testing)
--pipe mode spawns a demo MCP server and reads framed MCP messages from stdin (the same framing real MCP clients use). The simplest way to drive it is the included mcp-client:
# Start the gateway in pipe mode (spawns the demo server internally)
./proxymic --pipe
# In another terminal, send a tool call THROUGH the gateway
./mcp-client --proxy list_files path=/home
# Send the same call again → CACHE HIT
./mcp-client --proxy list_files path=/home
Configuration
Edit mcp-gateway.yaml — a single YAML file controls everything:
server:
command: "node"
args: ["server.js"]
timeout_seconds: 30
max_restarts: 5
cache:
ttl_seconds: 300
similarity_threshold: 0.95
security:
allowed_tools: ["list_files", "read_file", "write_file"]
blocked_tools: ["exec_command", "bash", "rm"]
rate_limit:
enabled: true
max_requests: 60
window_seconds: 60
circuit_breaker:
enabled: true
failure_threshold: 3
recovery_timeout_seconds: 30
loop_killer:
enabled: true
window_seconds: 30
block_seconds: 120
Project Structure
proxymic/
├── main.go # Entry point, CLI flags (--demo, --pipe, --config)
├── proxy.go # MCP stdio proxy + 7-layer tool call pipeline
├── config.go # YAML config loader with defaults
├── mcp-gateway.yaml # Configuration file
├── mcp_protocol.go # JSON-RPC 2.0 types for MCP
├── cache.go # In-memory tool response cache with TTL
├── circuitbreaker.go # Failure threshold + recovery probe
├── dedup.go # Response deduplication tracker
├── agent_loop.go # Loop detection → throttle → block
├── ratelimit.go # Per-tool rate limiting
├── recovery.go # Panic recovery middleware
├── log.go # Structured JSON event logging
├── tracer.go # Tool call tracing + anomaly detection + alerts
├── demo_page.go # Interactive playground HTML (embedded)
├── demo_traffic.go # Synthetic traffic generator for --demo
├── metrics.go # HTTP server: /metrics, /health, /api/call
├── railway.json # Railway deployment config
├── Dockerfile # Multi-stage Docker build
├── docker-compose.yml # Docker Compose for local demo
└── cmd/
├── mcp-client/ # Test client to send MCP calls via stdin
└── mcp-demo-server/ # Synthetic MCP server for demos
Built entirely in Go. No runtime dependencies. Single static binary.
Technical Details
- Transport: Intercepts stdio-based MCP protocol (JSON-RPC 2.0)
- Caching: HNSW vector index (local) + optional Redis (distributed) + L1 hot cache
- Embeddings: Local — no external embedding API needed. Uses a lightweight TF-IDF + hash-based embedding
- Loop detection: Sliding window with configurable timeframes per stage
- Deduplication: SHA-256 response hashing with session-scoped tracking
- Metrics: Prometheus
/metricsendpoint + JSON structured logs - Binary size: ~15MB static binary
License
MIT