84% fewer tokens on JSON tool results. 53% fewer tokens on tool-heavy requests. Sub-300ms semantic cache hits. Zero code changes.
Self-hosted LLM gateway and tier-routing proxy for Claude Code, Cursor, and Codex. Routes across Ollama, AWS Bedrock, OpenRouter, Databricks, Azure OpenAI, llama.cpp, and LM Studio with prompt caching, MCP tools, and 60-80% cost savings.
2026-11-15
Estrelas
—
Ferramentas
—
Disponibilidade
—
Numbers from the bundled benchmark against LiteLLM on identical free local backends — run it yourself: node benchmark-tier-routing.js. It doubles as a 19-scenario routing regression harness (currently 12/12 correctness checks), and MODE=routing runs a routing-only head-to-head that judges both proxies on the same acceptable-tier sets — including LiteLLM's Auto Router v2. How it works →
Agentic benchmark (Terminal-Bench core, 80 tasks, Oct 2026): Lynkr 42/80 for $1.54 vs openrouter/auto 44/80 for $4.72 on the same VM and evening — tied on accuracy within single-run noise (Lynkr's three runs: 39, 39, 42) at ~30% of the cost; vLLM Semantic Router on the same models/hosts: 25/80 (34 agent timeouts from unbounded upstream stalls and per-turn CPU classification). Full comparison, fairness notes and what it changed in Lynkr →
Third-party benchmark: on RouterArena (ICLR 2026, 8,400 queries) Lynkr's routing scores 67.65 arena / 68.41% accuracy at $0.29 per 1K queries with 92.38 robustness — above GPT-5's built-in router and NotDiamond at a fraction of their cost. Methodology & caveats →
🚀 New: Wrap Mode for AI Coding Tools
Use Lynkr's routing with your AI coding assistant — maximize your subscription value:
npm install -g lynkr
# Claude Code Pro/Max
lynkr wrap claude
Wrapping gives you:
✅ Tier routing (send simple tasks to free Ollama, complex to your subscription/API)
✅ Sticky sessions: one routing decision per conversation via content fingerprinting, with automatic escalation when the task outgrows the model
✅ TOON/RTK compression (84% token reduction on large JSON tool outputs)
✅ Your subscription is used only where it counts — simple requests route to your own local/API models, so subscription capacity goes to the work that needs it
✅ Works with OAuth (Claude, Copilot, Cursor) or API keys (Aider, Codex)
It asks four questions — usage mode (Claude Pro/Max via wrap, or direct API keys), tier picks for SIMPLE/MEDIUM/COMPLEX/REASONING across the 14 supported providers, credentials for what you chose, and a few routing-intelligence knobs — then writes a fully-populated .env with sensible production defaults for everything else (caching, compression, policy budgets, MCP sandbox, rate limiting).
✅ 84% compression on large JSON tool results (grep, file reads, test output)
✅ Semantic cache serves repeated queries in under 300ms with 0 tokens billed
✅ Automatic tier routing — simple questions go to cheap models, complex ones escalate; sessions stick to one model until the task genuinely outgrows it
✅ A closed learning loop — every outcome trains a kNN router and bandit, and tier thresholds re-calibrate nightly from your own traffic
✅ Route through your company's infrastructure (Databricks, Azure, Bedrock)
✅ Zero code changes — just change one environment variable
Supported Providers
Provider
Type
Example Models
Cost
Ollama
Local
qwen2.5-coder, deepseek-coder, llama3
Free
llama.cpp
Local
Any GGUF model
Free
LM Studio
Local
Local models with GUI
Free
OpenRouter
Cloud
GPT-4o, Claude 3.5, Llama 3, Gemini
$
OrcaRouter
Cloud
Chat, reasoning, image, video (one endpoint, live catalog)
$ (zero-markup)
AWS Bedrock
Cloud
Claude, Llama, Mistral, Titan
$$
Databricks
Cloud
Claude Sonnet 4.5, Opus 4.6
$$$
Azure OpenAI
Cloud
GPT-4o, o1, o3
$$$
Azure Anthropic
Cloud
Claude Sonnet, Opus
$$$
OpenAI
Cloud
GPT-4o, o3-mini
$$$
Atlas Cloud
Cloud
Qwen, DeepSeek, and other OpenAI-compatible models
$-$$$
Cheaper Inference
Cloud
GPT, Claude, Gemini, DeepSeek, and other OpenAI-compatible models
$-$$$
AtmoRouter
Cloud
Claude, GPT, Gemini, GLM, Kimi, Qwen, DeepSeek, and other OpenAI-compatible models
$-$$
DeepSeek
Cloud
DeepSeek R1, Reasoner
$
Z.ai
Cloud
GLM-4.7, GLM-4.5-Air
$
Moonshot AI
Cloud
Kimi K2.6, Kimi K3
$
Baidu Qianfan
Cloud
ERNIE 4.5 Turbo, ERNIE X1.1
$ (unverified — not yet probed against a live key)
Fireworks AI
Cloud
Kimi K2, GLM-5, DeepSeek V3 (serverless)
$ (unverified — not yet probed against a live key)
4 local providers for 100% offline, free usage. 14+ cloud providers for scale.
Advanced: Tier Routing (Save Even More)
Route different request types to different models automatically:
# .env file
MODEL_PROVIDER=ollama
FALLBACK_ENABLED=false
# Use small/fast models for simple tasks
TIER_SIMPLE=ollama:qwen2.5:3b
# Use medium models for normal coding
TIER_MEDIUM=ollama:qwen2.5:7b
# Use powerful models for complex architecture
TIER_COMPLEX=ollama:deepseek-r1:14b
TIER_REASONING=ollama:deepseek-r1:14b
# Optional: Limits (remove for unlimited) for long conversations
POLICY_MAX_STEPS=50
POLICY_MAX_TOOL_CALLS=100
Lynkr analyzes each request and routes it to the appropriate tier. Simple questions use fast models. Complex refactoring uses powerful models. The scorer combines anchor-embedding classification (WS7, payload-invariant) with the Jev routing judge (an optional typed second opinion over the four tiers, enabled by TYPESAFE_API_KEY; it replaced the earlier local qwen2.5:3b classifier, so no local Ollama is required) that catches topic-vs-difficulty confounding — list the exports from this file correctly routes to MEDIUM instead of getting boosted to REASONING by "technical vocabulary" alone.
Result: 70-90% of requests use cheaper/faster models. Only hard problems hit expensive models. 15× reduction in expensive-tier over-routing vs the anchor-only baseline on the 381-prompt eval set (0.6% vs ~15%).
Tier configuration is strictly authoritative — bandit exploration is constrained to the models you've listed in TIER_*, and multi-turn conversations score with a recency-weighted sliding window so context isn't lost on short follow-ups. Conversations get a content-fingerprint session id (clients like Claude Code send none), the decision pins for the session, and a guarded escape ladder (risk keywords, force phrases, score drift, context overflow) re-escalates the moment a task outgrows its model. Full pipeline: docs/routing-intelligence.md · intent scorer: docs/intent-window-routing.md · verify any change: docs/benchmarking.md.
Common Issues & Fixes
Issue
Solution
"Service temporarily overloaded"
Ollama model too large for RAM. Use smaller model or increase --max-old-space-size
"Route not found: HEAD /"
Ignore - harmless health check from Claude Code
"Hallucinated tool calls"
Normal - Lynkr automatically filters invalid tools
"Safe Command DSL blocked"
Add POLICY_SAFE_COMMANDS_ENABLED=false to .env
"spawn graphify ENOENT"
Optional feature. Set CODE_GRAPH_ENABLED=false in .env (see Advanced Features section for installation)
Slow first request (20+ sec)
Ollama loading model into memory. Add OLLAMA_KEEP_ALIVE=30m in Ollama config
No response after N turns
Remove POLICY_MAX_STEPS and POLICY_MAX_TOOL_CALLS from .env (unlimited by default in v9.3.0+)
Advanced Features
Real-time SSE streaming (all tiers)
Responses stream token-by-token instead of arriving all at once — including
through the tier router. Two mechanisms, both on by default:
Native passthrough: when the upstream already speaks Anthropic SSE
(Anthropic endpoints, Z.AI, Ollama v0.14+), Lynkr pipes the bytes straight
through with backpressure. Kill switch: LYNKR_NATIVE_PASSTHROUGH=false.
Cross-format transform: OpenAI-format upstreams (OpenAI, Azure OpenAI,
OpenRouter, Databricks, llama.cpp, LM Studio) are reshaped into Anthropic
events in flight — including reassembling tool-call argument fragments into
clean tool_use blocks. Kill switch: LYNKR_STREAM_TRANSFORM=false.
The LYNKR_VISIBLE_ROUTING badge streams too (injected as the first content
block), and telemetry is recorded on stream close with real token counts.
Fallback safety: if an upstream fails before the first byte, the request
falls back to the buffered path automatically.
# Ollama thinking models (MiniMax): streaming skips the <think>-leak repair.
# If you see raw <think> text in responses, buffer that provider instead:
LYNKR_OLLAMA_BUFFER_RESPONSES=true # default true; false = stream Ollama
Token Optimization (60-80% savings)
# Enable all optimizations
PROMPT_CACHE_ENABLED=true
SEMANTIC_CACHE_ENABLED=true
Always-on (no config): RTK tool-result compression
(test/git/grep/lint/build/JSON output), MCP tool dedup (drops
built-in WebSearch/WebFetch when an Exa/Tavily MCP tool is present), and
request bypass (Claude CLI Warmup / title-extraction calls are answered
locally, never hitting a provider).
Optional terse-output mode to cut output tokens:
CAVEMAN_ENABLED=true # off by default — nudges the model to be concise
CAVEMAN_LEVEL=lite # lite | full | ultra
Built-in dashboard
Open http://localhost:8081/dashboard while Lynkr is running. No setup, reads
the local telemetry store:
Spend & savings — actual cost vs what the same traffic would have cost
on the flagship model (the counterfactual no pass-through gateway can show,
since they neither pick your model nor see local-model traffic)
Tier mix — daily request breakdown across SIMPLE/MEDIUM/COMPLEX/REASONING
Routing accuracy — over-/under-provisioned request counts, a self-audit
of the tier router's decisions
Request logs — filterable by provider, tier, and errors, with latency,
tokens, and cost per request; every session id links to a drill-down
Provider health — configured providers, credential warnings, circuit
breaker states
Insights — analyzer findings ranked by past overspend (auditable
against spend already recorded, never projected savings): dead tool
schemas paying rent, harness side-traffic served on expensive tiers,
weak prompt-cache hit ratios, and Wilson-bounded tier-downsize verdicts.
Each finding ships its evidence rows, the fix, the caveat, and how the
figure was computed. "Not measured yet" is rendered distinctly from
zero — an un-run scan is not an all-clear.
Explore — a Metric × Dimension (× Stack) pivot over routing telemetry
(spend/tokens/requests/sessions by provider, model, tier, request type,
day…) with preset views, period-over-period deltas, and CSV export
Session drill-down — per-model cost mix and an input-context-per-call
chart where distillation and cache behavior are directly visible
Cache receipts — projected switch/hold economics from cache-aware
routing next to the measured companion (dollars saved by cache reads
actually served)
JSON APIs behind it
(/dashboard/api/overview|usage|routing|logs|recommendations|analytics|sessions/:id|statusline)
if you want the raw numbers.
Status line for Claude Code
One line after every turn — routed tier, served model, today's spend, and
cache re-read share — with zero token cost (out-of-band, never enters model
context). Add to ~/.claude/settings.json:
◆ Opus 4.8 · MEDIUM → minimax-m3:cloud (ollama) · today $0.06 · cache 67%
Always prints exactly one line and always exits 0 — a down proxy can never
break the harness. Set LYNKR_PORT if Lynkr isn't on 8081.
Cost tracking & model pricing
Per-request cost is computed from a model-pricing registry (LiteLLM → models.dev,
cached 24h) and recorded in telemetry. Models the registry doesn't know record
cost_usd=null (logged once) rather than a fabricated price. Pin prices for
unknown models:
# Per-1M-token USD prices, JSON keyed by model name
MODEL_PRICE_OVERRIDES={"my-model":{"input":0.5,"output":1.5}}
Head-to-head against LiteLLM on the same backends (Ollama minimax-m2.5, Moonshot, Azure OpenAI), 9 scenarios across 4 feature categories. Apples-to-apples comparison is Lynkr vs LiteLLM billed tokens on the same scenario. Run with node benchmark-tier-routing.js.
Runs: token benchmarks — June 5, 2026 (Lynkr v9.3.2 · LiteLLM v1.87.1). Tier routing head-to-head — re-run July 19, 2026 (Lynkr working tree w/ Phase-6 classifier + config B · LiteLLM v1.94.0.dev1 Auto Router v2). macOS, Apple Silicon.
Token reduction (vs LiteLLM, same model & prompt)
Mechanism
Lynkr
LiteLLM
Result
TOON compression (60-item grep JSON)
427 tokens · $0.009
3,458 tokens · $0.018
87.6% fewer tokens, 50% cheaper
Lynkr binary-compresses large JSON tool results (TOON) in-process, with no added latency.
Semantic cache
Tokens billed
Response time
First call (cold)
2,857
1,891ms
Second call — paraphrased, cache hit
0 (served from cache)
171ms (11× faster)
Near-identical prompts return cached responses in 171ms. Zero model tokens billed on a cache hit.
Tier routing — vs LiteLLM Auto Router v2 (re-run July 19, 2026)
LiteLLM v1.94 shipped a native complexity router (auto_router/complexity_router) with the same four tier names Lynkr uses. Head-to-head on the same backends with identical prompts, both proxies live, both judged against the same acceptable-tier sets (11 routing scenarios, MODE=routing node benchmark-tier-routing.js, LiteLLM config in litellm-autorouter-v2.yaml):
Anchor-embedding classifier over 4 difficulty classes + FRONTIER_MIN_SIM floor
Jev routing judge (optional, TYPESAFE_API_KEY) reconciles borderline anchor calls — no local Ollama required; the earlier qwen2.5:3b classifier is kept dark
Envelope invariance — scores cleaned user text only, ignoring tool schemas / history / system-reminders
LiteLLM's router reads the raw last message with no envelope-stripping, no verify-then-escalate cascade, no risk classifier, no agentic detection — its fallbacks trigger only on HTTP errors, never on a bad answer.
Fairness notes: the 11 scenarios derive from Lynkr's own regression suite, so Lynkr has home-field advantage — the transferable finding is the direction of LiteLLM's failures (systematic under-routing to cheap on hard prompts), not the exact scores. Lynkr's top tiers used Claude Opus 4.8 / z.ai GLM-5.2; LiteLLM was given the identical tier targets.
Cost projection (100,000 requests/month, same backend)
Direct cost (raw meter):
Monthly cost
vs LiteLLM
LiteLLM (TOON tool-heavy scenario)
~$818
baseline
Lynkr (TOON tool-heavy)
~$409
~50% cheaper via token optimization
Effective cost per correct-tier answer (July 19 head-to-head, all 11 routing scenarios):
Direct $/mo
Effective $/mo if you re-issue misroutes at correct tier
Lynkr
~$5.50 (spends where it should, doesn't where it shouldn't)
~$5.50
LiteLLM v2 heuristic
$0 (under-routes everything to ollama)
~$89 (7 misroutes need re-issue on Moonshot/COMPLEX)
The story isn't "Lynkr always spends less." It's Lynkr spends where it should and doesn't where it shouldn't. LiteLLM's zero-dollar cost on complex prompts is bought with wrong-tier answers to hard work — a 7B model answering "prove this rate limiter is fair under concurrent refill" is silent quality collapse, not a savings.
✅ evidence-backed findings (dead tool schemas, side-traffic, cache weak spots, Wilson-proven downsizes) ranked by past overspend
❌
❌
⚠️ alerts, no findings
Long-term memory
✅ SQLite, per-session
❌
❌
❌
MCP integration
✅
❌
❌
❌
Self-hosted
✅ Node.js only
✅ Python stack
❌ SaaS
✅ Docker
Dependencies
Node.js 20+
Python, Prisma, PostgreSQL
None
Docker, Python
Lynkr's edge: Purpose-built for AI coding tools. Compresses tokens before they reach the model — not just after. Zero-config for Claude Code, Cursor, and Codex. Installs in one command.
Self-hosted LLM gateway and tier-routing proxy for Claude Code, Cursor, and Codex. Routes across Ollama, AWS Bedrock, OpenRouter, Databricks, Azure OpenAI, llama.cpp, and LM Studio with prompt caching
Self-hosted LLM gateway and tier-routing proxy for Claude Code, Cursor, and Codex. Routes across Ollama, AWS Bedrock, OpenRouter, Databricks, Azure OpenAI, llama.cpp, and LM Studio with prompt caching, MCP tools, and 60-80% cost savings.
Ferramentas
Ferramentas
Ainda não há lista de ferramentas. Servidores remotos recebem uma após a primeira verificação de saúde.
Perguntas frequentes
Perguntas
Como instalo Lynkr?
Copie nesta página a configuração stdio para Cursor, Claude Desktop, Claude Code ou VS Code. Ela usa a instalação do publicador, não um proxy do PlugMCP.
Este registro não lista nenhuma variável de ambiente obrigatória.
Qual transporte Lynkr usa?
Lynkr roda localmente via stdio.
Quando Lynkr foi verificado pela última vez?
Última verificação: 2026-11-15. Self-hosted LLM gateway and tier-routing proxy for Claude Code, Cursor, and Codex. Routes across Ollama, AWS Bedrock, OpenRouter, Databricks, Azure OpenAI, llama.cpp, and LM Studio with prompt caching, MCP tools, and 60-80% cost savings.