CascadeGateway
jadberro4 estrellas
Hardware-adaptive local LLM & cloud cascading gateway. Sub-5ms intelligent routing, 3-token lookahead failover, asymmetric verification, 1-click IDE config, and MCP for $0 token cost.
- Transporte
- stdio
- Estado
- Local
Potencia tu agente
Instalación del publicador. No es un endpoint de PlugMCP.
Host a copy with our keys and logs.
Aún no hay una versión empaquetada en el registro. Sigue el README del repositorio.
¿URL remota o un comando en tu máquina? En qué se diferencian las dos instalaciones
Ejecutar en PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
AbrirCascadeGateway is an intelligent, hardware-adaptive cascading proxy that routes AI queries between local GPUs (via Ollama at $0 token cost) and frontier cloud models (Google Gemini, OpenAI, etc.).
It automatically detects your GPU hardware and physical VRAM on startup—whether running a flagship RTX 5090 (32GB), RTX 4090 (24GB), mainstream RTX 3080 (10GB), or Apple Silicon—and dynamically selects and sizes the optimal models without requiring manual reconfiguration.
flowchart TD
Client["AI Client: Cursor / Continue / Antigravity"] --> Gateway["CascadeGateway :8000/v1"]
Gateway --> Phase1{"Phase 1: Structural Guard"}
Phase1 -->|"Context > 32k or Tools"| CloudFallback["Cloud Fallback: Gemini"]
Phase1 -->|"Passed"| Phase2{"Phase 2: Lexical Automaton <1ms"}
Phase2 -->|"Architecture / Reasoning"| LocalArchitect["Local Architect: DeepSeek-R1 / Gemma 4 (RTX 5090)"]
Phase2 -->|"Coding / Tests / Routine"| LocalBuilder["Local Builder: Qwen 2.5 Coder 32B (RTX 5090)"]
LocalArchitect --> Phase3{"Phase 3: 3-Token Lookahead"}
LocalBuilder --> Phase3
Phase3 -->|"Stalled / Timeout"| CloudFallback
Phase3 -->|"Healthy Stream"| LoopBreaker["Sliding-Window Loop Breaker"]
LoopBreaker --> ClientStream["SSE Stream to Client"]
CloudFallback --> ClientStream
VerifyRoute["Asymmetric Verification: /verify"] --> LocalDraft["Local Generator: RTX 5090"]
LocalDraft --> CloudCritic["Cloud Critic: Gemini Flash"]
CloudCritic --> ClientStream
Gateway -. Telemetry .-> HUD["🖥️ Always-On-Top Desktop HUD"]
#0b0f19) that sits above your IDEs next to the clock or anywhere on screen. Displays live routing decisions (🟢 Local Builder, 🟣 Local Architect, 🔵 Asymmetric Verify, 🟠 Gemini Cloud), real-time VRAM gauge (GB, %, °C, W), decision latency (sub-5ms), prompt snippet, dollar savings counter, pin toggle, compact mini-mode, and 1-click mode switcher buttons.deepseek-r1:14b or gemma4:26b) on the RTX 5090 rather than escaping to cloud tokens. Routine code generation is routed to qwen2.5-coder:32b.nvidia-smi / Metal / CPU) and assigns the optimal model tier.qwen2.5-coder:32b for quick functions, tests, and scripts./verify): High-speed local draft synthesis on RTX 5090 ($0) combined with an anonymous strict cloud auditor (Gemini) providing production-grade critique and code enhancement without leaking full history./api/ide/auto-config) and UI button for VSCode Continue (~/.continue/config.json) and Cursor.eval_count) calculating real-time dollar savings based on commercial frontier rates ($3.00/1M tokens).keep_alive: -1) across idle periods and tasks, ensuring instant zero-delay responses. Includes manual 1-Click "Pause GPU" (instant VRAM purge for AAA gaming) and "Warm GPU" (preloads weights into VRAM).deepseek-r1:14b, gemma4:26b, etc.) and the Builder role (qwen2.5-coder:32b, etc.) directly from the Web UI toolbar or Windows Tray submenus. Automatically detects newly pulled models from Ollama without restarting the gateway.POST /api/models/unload. Frees 20–30+ GB of VRAM immediately for AAA gaming, Blender, or video editing without terminating the server. Models reload automatically on demand when coding./v1/chat/completions) for Cursor, VSCode (Continue.dev), Aider, Claude Dev, and custom scripts.sequenceDiagram
autonumber
actor Dev as You (Developer)
participant Arch as 🧠 Architect - DeepSeek-R1 / Gemma 4
participant Gate as 🛑 Review Gate - You in the Loop
participant Build as ⚡ Builder - Qwen 2.5 Coder 32B
Dev->>Arch: "Add rate-limiting and burst protection to API routes"
Note over Arch: Deep CoT: Identifies race conditions,<br/>evaluates Redis vs in-memory,<br/>drafts interface contracts.
Arch->>Gate: Presents Architectural Blueprint + Edge Cases
Note over Gate: PAUSE: No code written yet.<br/>You review the proposed interfaces & strategy.
alt If you want adjustments
Dev->>Gate: "Use Redis, and add IPv6 CIDR subnet matching"
Gate->>Arch: Quick amendment - 100 tokens
Arch->>Gate: Updated spec
end
Dev->>Gate: "Approve & Build"
Gate->>Build: Sends final structured specification
Note over Build: Zero ambiguity.<br/>High-speed code synthesis - 70 t/s.
Build-->>Dev: Delivers complete implementation + unit tests - $0 Cost
When running agentic coding assistants like Google Antigravity on a Pro plan, repeatedly shipping 20k to 50k tokens of repo file dumps, compiler logs, and multi-turn message history quickly exhausts 5-hour quota refill blocks.
CascadeGateway implements the native Hybrid Architect-Builder Pattern:
flowchart TD
subgraph Cloud ["☁️ Cloud Tier (Pro Plan Quota)"]
Architect["Cloud Architect: Gemini Pro / Claude<br/>• High-level system decomposition<br/>• Milestone roadmap planning<br/>• Concise diff reviews"]
end
subgraph Gateway ["⚡ CascadeGateway :8000/v1"]
Router["Context Pruning & Intent Router"]
Clamp["Terminal Clamping (>60 lines → 15 head + 30 tail)"]
Dedup["Repeat File Buffer Deduplication"]
end
subgraph Local ["🖥️ Local Workstation Tier (NVIDIA RTX 5090 - 32GB VRAM)"]
Builder["Local Builder (vLLM / Ollama): Qwen 2.5 Coder 32B<br/>• Code generation, unit tests, refactoring<br/>• Iterative bug fixes & lint diagnostics<br/>• $0 Cloud Token Cost"]
LocalMCP["Local MCP Tools<br/>• AST Symbol Lookup (@llmtooling/code-search-mcp)<br/>• Targeted File Slicing"]
end
Architect -->|"Milestone Specifications"| Router
Router --> Clamp
Clamp --> Dedup
Dedup -->|"Iterative Coding Tasks"| Builder
Builder <-->|"Precise Symbol Queries"| LocalMCP
Dedup -. Escalation on repeated test failures .-> Architect
Benchmarked using python -m cascadegateway.audit --compare baseline.jsonl optimized.jsonl:
Windows Native (Zero-Window Flash):
powershell -File scripts\start_local_engine.ps1
Auto-detects Ollama or vLLM, warms up qwen2.5-coder:32b in 32GB VRAM with keep_alive: -1.
Linux / WSL (vLLM with Automatic Prefix Caching):
./scripts/start_local_vllm.sh
Launches vllm with --enable-prefix-caching, --enable-chunked-prefill, and --tool-call-parser hermes.
Antigravity SDK & Standalone Orchestration:
python scripts/run_architect_builder.py --task "Draft and build target module"
git clone https://github.com/jadberro/CascadeGateway.git
cd CascadeGateway
scripts\setup.bat
Creates .venv, installs dependencies, auto-generates your desktop shortcut, and places an auto-start shortcut in your Windows Startup menu.
git clone https://github.com/jadberro/CascadeGateway.git
cd CascadeGateway
chmod +x scripts/setup.sh
./scripts/setup.sh
Once running, the control center is available at http://127.0.0.1:8000/.
Add to ~/.gemini/config/mcp_config.json:
{
"mcpServers": {
"rtx5090-cascade": {
"serverUrl": "http://127.0.0.1:8000/mcp"
}
}
}
Antigravity automatically detects the toolset (query_local_5090, cascade_llm, get_cascade_metrics) and dynamically dispatches code generation directly to your GPU.
http://127.0.0.1:8000/v1cascading-local (any string)cascade-autoconfig.json){
"models": [
{
"title": "Cascade Gateway (Local + Cloud)",
"provider": "openai",
"model": "cascade-auto",
"apiBase": "http://127.0.0.1:8000/v1",
"apiKey": "cascading-local"
}
],
"tabAutocompleteModel": {
"title": "Local Autocomplete",
"provider": "openai",
"model": "cascade-auto",
"apiBase": "http://127.0.0.1:8000/v1",
"apiKey": "cascading-local"
}
}
claude_desktop_config.json){
"mcpServers": {
"local-cascade": {
"serverUrl": "http://127.0.0.1:8000/mcp"
}
}
}
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8000/v1",
api_key="cascading-local"
)
response = client.chat.completions.create(
model="cascade-auto",
messages=[
{"role": "user", "content": "Write an optimized LRU cache in Python."}
]
)
print(response.choices[0].message.content)
CascadeGateway is structured as a modern, modular Python package:
CascadeGateway/
├── src/
│ └── cascadegateway/
│ ├── core/ # Hardware profiler, model sizing & routing engine
│ │ ├── config.py # YAML config & environment loader
│ │ ├── hardware.py # GPU VRAM auto-profiling (5090/4090/etc.)
│ │ ├── classifier.py # Phase 1 structural validation & Phase 2 regex automaton (<1ms)
│ │ ├── streaming.py # Phase 3 lookahead failover, loop breaker & asymmetric audit
│ │ └── router.py # Biasing engine, metrics & model resolution
│ ├── api/ # Modular FastAPI endpoints
│ │ ├── server.py # App initialization, SSE streaming & /v1/chat/completions
│ │ ├── pipeline.py # Architect, Review Gate & Builder endpoints
│ │ └── models.py # Pause & Free GPU (VRAM purge) & 1-click IDE auto-config
│ ├── web/ # Clean decoupled web assets & responsive UI
│ │ ├── templates/
│ │ │ └── dashboard.html # Responsive HTML5 dashboard & real-time telemetry
│ │ └── static/
│ │ ├── css/dashboard.css
│ │ └── js/dashboard.js
│ ├── hud/ # Always-On-Top Desktop HUD & Telemetry Overlay
│ │ ├── __init__.py
│ │ └── overlay.py # Tkinter floating dark obsidian window
│ ├── tray/ # Windows system tray controller
│ │ └── app.py # Pystray background service & menus
│ ├── mcp/ # Model Context Protocol endpoints
│ │ ├── __init__.py
│ │ └── server.py # Antigravity & Claude Desktop integration
├── scripts/ # Platform launchers & utilities
│ ├── start_cascade.bat # Windows batch launcher
│ ├── start_tray.vbs # Silent windowless tray runner
│ ├── start_hud.bat # Desktop HUD launcher
│ ├── start_hud.vbs # Silent windowless HUD launcher
│ ├── setup.bat / setup.sh # One-click installers
│ └── create_shortcuts.ps1 # Desktop & startup shortcut generator
├── tests/ # Comprehensive automated test suites
│ ├── test_gateway.py # End-to-end API integration tests
│ ├── test_resilience.py # Sub-5ms SLA, structural validation & lookahead tests
│ └── test_new_features.py # Loop breaker, IDE auto-config, dollar savings & /verify tests
├── assets/ # Icons & diagrams
├── config.yaml # Gateway configuration
├── pyproject.toml # Modern pip/uv packaging metadata
└── requirements.txt
config.yaml)server:
host: "127.0.0.1"
port: 8000
local:
provider: "ollama"
base_url: "http://127.0.0.1:11434"
# Set empty to let the hardware profiler auto-select the best model for your GPU
primary_model: ""
max_context_tokens: 32768
timeout_seconds: 120
cloud:
provider: "gemini"
default_model: "gemini-2.5-flash"
pro_model: "gemini-2.5-pro"
max_context_tokens: 1048576
biasing:
mode: "adaptive" # "adaptive", "manual", or "local_only"
bias_factor: 0.35 # Baseline local bias (0.0 to 1.0)
cloud_token_budget: 500000
allow_context_overflow_to_cloud: true
This project is licensed under the MIT License.
Hardware-adaptive local LLM & cloud cascading gateway. Sub-5ms intelligent routing, 3-token lookahead failover, asymmetric verification, 1-click IDE config, and MCP for $0 token cost.
Herramientas
Aún no hay lista de herramientas. Los servidores remotos la reciben tras la primera comprobación de estado.
Preguntas frecuentes
Copia en esta página la configuración stdio para Cursor, Claude Desktop, Claude Code o VS Code. Usa la instalación del publicador, no un proxy de PlugMCP.
Este registro no indica ninguna variable de entorno obligatoria.
CascadeGateway se ejecuta localmente por stdio.
Última comprobación: 2026-10-28. Hardware-adaptive local LLM & cloud cascading gateway. Sub-5ms intelligent routing, 3-token lookahead failover, asymmetric verification, 1-click IDE config, and MCP for $0 token cost.