CascadeGateway
jadberro4 ดาว
Hardware-adaptive local LLM & cloud cascading gateway. Sub-5ms intelligent routing, 3-token lookahead failover, asymmetric verification, 1-click IDE config, and MCP for $0 token cost.
- ทรานสปอร์ต
- stdio
- สถานะ
- โลคัล
Power your agent
การติดตั้งจากผู้เผยแพร่ ไม่ใช่เอนด์พอยต์ของ PlugMCP
URL แบบรีโมตหรือคำสั่งบนเครื่องของคุณ? การติดตั้งสองแบบต่างกันอย่างไร
รันบน PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
เปิดCascadeGateway is an intelligent, hardware-adaptive cascading proxy that routes AI queries between local GPUs (via Ollama at $0 token cost) and frontier cloud models (Google Gemini, OpenAI, etc.).
It automatically detects your GPU hardware and physical VRAM on startup—whether running a flagship RTX 5090 (32GB), RTX 4090 (24GB), mainstream RTX 3080 (10GB), or Apple Silicon—and dynamically selects and sizes the optimal models without requiring manual reconfiguration.
flowchart TD
Client["AI Client: Cursor / Continue / Antigravity"] --> Gateway["CascadeGateway :8000/v1"]
Gateway --> Phase1{"Phase 1: Structural Guard"}
Phase1 -->|"Context > 32k or Tools"| CloudFallback["Cloud Fallback: Gemini"]
Phase1 -->|"Passed"| Phase2{"Phase 2: Lexical Automaton <1ms"}
Phase2 -->|"Architecture / Reasoning"| LocalArchitect["Local Architect: DeepSeek-R1 / Gemma 4 (RTX 5090)"]
Phase2 -->|"Coding / Tests / Routine"| LocalBuilder["Local Builder: Qwen 2.5 Coder 32B (RTX 5090)"]
LocalArchitect --> Phase3{"Phase 3: 3-Token Lookahead"}
LocalBuilder --> Phase3
Phase3 -->|"Stalled / Timeout"| CloudFallback
Phase3 -->|"Healthy Stream"| LoopBreaker["Sliding-Window Loop Breaker"]
LoopBreaker --> ClientStream["SSE Stream to Client"]
CloudFallback --> ClientStream
VerifyRoute["Asymmetric Verification: /verify"] --> LocalDraft["Local Generator: RTX 5090"]
LocalDraft --> CloudCritic["Cloud Critic: Gemini Flash"]
CloudCritic --> ClientStream
Gateway -. Telemetry .-> HUD["🖥️ Always-On-Top Desktop HUD"]
#0b0f19) that sits above your IDEs next to the clock or anywhere on screen. Displays live routing decisions (🟢 Local Builder, 🟣 Local Architect, 🔵 Asymmetric Verify, 🟠 Gemini Cloud), real-time VRAM gauge (GB, %, °C, W), decision latency (sub-5ms), prompt snippet, dollar savings counter, pin toggle, compact mini-mode, and 1-click mode switcher buttons.deepseek-r1:14b or gemma4:26b) on the RTX 5090 rather than escaping to cloud tokens. Routine code generation is routed to qwen2.5-coder:32b.nvidia-smi / Metal / CPU) and assigns the optimal model tier.qwen2.5-coder:32b for quick functions, tests, and scripts./verify): High-speed local draft synthesis on RTX 5090 ($0) combined with an anonymous strict cloud auditor (Gemini) providing production-grade critique and code enhancement without leaking full history./api/ide/auto-config) and UI button for VSCode Continue (~/.continue/config.json) and Cursor.eval_count) calculating real-time dollar savings based on commercial frontier rates ($3.00/1M tokens).keep_alive: -1) across idle periods and tasks, ensuring instant zero-delay responses. Includes manual 1-Click "Pause GPU" (instant VRAM purge for AAA gaming) and "Warm GPU" (preloads weights into VRAM).deepseek-r1:14b, gemma4:26b, etc.) and the Builder role (qwen2.5-coder:32b, etc.) directly from the Web UI toolbar or Windows Tray submenus. Automatically detects newly pulled models from Ollama without restarting the gateway.POST /api/models/unload. Frees 20–30+ GB of VRAM immediately for AAA gaming, Blender, or video editing without terminating the server. Models reload automatically on demand when coding./v1/chat/completions) for Cursor, VSCode (Continue.dev), Aider, Claude Dev, and custom scripts.sequenceDiagram
autonumber
actor Dev as You (Developer)
participant Arch as 🧠 Architect - DeepSeek-R1 / Gemma 4
participant Gate as 🛑 Review Gate - You in the Loop
participant Build as ⚡ Builder - Qwen 2.5 Coder 32B
Dev->>Arch: "Add rate-limiting and burst protection to API routes"
Note over Arch: Deep CoT: Identifies race conditions,<br/>evaluates Redis vs in-memory,<br/>drafts interface contracts.
Arch->>Gate: Presents Architectural Blueprint + Edge Cases
Note over Gate: PAUSE: No code written yet.<br/>You review the proposed interfaces & strategy.
alt If you want adjustments
Dev->>Gate: "Use Redis, and add IPv6 CIDR subnet matching"
Gate->>Arch: Quick amendment - 100 tokens
Arch->>Gate: Updated spec
end
Dev->>Gate: "Approve & Build"
Gate->>Build: Sends final structured specification
Note over Build: Zero ambiguity.<br/>High-speed code synthesis - 70 t/s.
Build-->>Dev: Delivers complete implementation + unit tests - $0 Cost
When running agentic coding assistants like Google Antigravity on a Pro plan, repeatedly shipping 20k to 50k tokens of repo file dumps, compiler logs, and multi-turn message history quickly exhausts 5-hour quota refill blocks.
CascadeGateway implements the native Hybrid Architect-Builder Pattern:
flowchart TD
subgraph Cloud ["☁️ Cloud Tier (Pro Plan Quota)"]
Architect["Cloud Architect: Gemini Pro / Claude<br/>• High-level system decomposition<br/>• Milestone roadmap planning<br/>• Concise diff reviews"]
end
subgraph Gateway ["⚡ CascadeGateway :8000/v1"]
Router["Context Pruning & Intent Router"]
Clamp["Terminal Clamping (>60 lines → 15 head + 30 tail)"]
Dedup["Repeat File Buffer Deduplication"]
end
subgraph Local ["🖥️ Local Workstation Tier (NVIDIA RTX 5090 - 32GB VRAM)"]
Builder["Local Builder (vLLM / Ollama): Qwen 2.5 Coder 32B<br/>• Code generation, unit tests, refactoring<br/>• Iterative bug fixes & lint diagnostics<br/>• $0 Cloud Token Cost"]
LocalMCP["Local MCP Tools<br/>• AST Symbol Lookup (@llmtooling/code-search-mcp)<br/>• Targeted File Slicing"]
end
Architect -->|"Milestone Specifications"| Router
Router --> Clamp
Clamp --> Dedup
Dedup -->|"Iterative Coding Tasks"| Builder
Builder <-->|"Precise Symbol Queries"| LocalMCP
Dedup -. Escalation on repeated test failures .-> Architect
Benchmarked using python -m cascadegateway.audit --compare baseline.jsonl optimized.jsonl:
Windows Native (Zero-Window Flash):
powershell -File scripts\start_local_engine.ps1
Auto-detects Ollama or vLLM, warms up qwen2.5-coder:32b in 32GB VRAM with keep_alive: -1.
Linux / WSL (vLLM with Automatic Prefix Caching):
./scripts/start_local_vllm.sh
Launches vllm with --enable-prefix-caching, --enable-chunked-prefill, and --tool-call-parser hermes.
Antigravity SDK & Standalone Orchestration:
python scripts/run_architect_builder.py --task "Draft and build target module"
git clone https://github.com/jadberro/CascadeGateway.git
cd CascadeGateway
scripts\setup.bat
Creates .venv, installs dependencies, auto-generates your desktop shortcut, and places an auto-start shortcut in your Windows Startup menu.
git clone https://github.com/jadberro/CascadeGateway.git
cd CascadeGateway
chmod +x scripts/setup.sh
./scripts/setup.sh
Once running, the control center is available at http://127.0.0.1:8000/.
Add to ~/.gemini/config/mcp_config.json:
{
"mcpServers": {
"rtx5090-cascade": {
"serverUrl": "http://127.0.0.1:8000/mcp"
}
}
}
Antigravity automatically detects the toolset (query_local_5090, cascade_llm, get_cascade_metrics) and dynamically dispatches code generation directly to your GPU.
http://127.0.0.1:8000/v1cascading-local (any string)cascade-autoconfig.json){
"models": [
{
"title": "Cascade Gateway (Local + Cloud)",
"provider": "openai",
"model": "cascade-auto",
"apiBase": "http://127.0.0.1:8000/v1",
"apiKey": "cascading-local"
}
],
"tabAutocompleteModel": {
"title": "Local Autocomplete",
"provider": "openai",
"model": "cascade-auto",
"apiBase": "http://127.0.0.1:8000/v1",
"apiKey": "cascading-local"
}
}
claude_desktop_config.json){
"mcpServers": {
"local-cascade": {
"serverUrl": "http://127.0.0.1:8000/mcp"
}
}
}
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8000/v1",
api_key="cascading-local"
)
response = client.chat.completions.create(
model="cascade-auto",
messages=[
{"role": "user", "content": "Write an optimized LRU cache in Python."}
]
)
print(response.choices[0].message.content)
CascadeGateway is structured as a modern, modular Python package:
CascadeGateway/
├── src/
│ └── cascadegateway/
│ ├── core/ # Hardware profiler, model sizing & routing engine
│ │ ├── config.py # YAML config & environment loader
│ │ ├── hardware.py # GPU VRAM auto-profiling (5090/4090/etc.)
│ │ ├── classifier.py # Phase 1 structural validation & Phase 2 regex automaton (<1ms)
│ │ ├── streaming.py # Phase 3 lookahead failover, loop breaker & asymmetric audit
│ │ └── router.py # Biasing engine, metrics & model resolution
│ ├── api/ # Modular FastAPI endpoints
│ │ ├── server.py # App initialization, SSE streaming & /v1/chat/completions
│ │ ├── pipeline.py # Architect, Review Gate & Builder endpoints
│ │ └── models.py # Pause & Free GPU (VRAM purge) & 1-click IDE auto-config
│ ├── web/ # Clean decoupled web assets & responsive UI
│ │ ├── templates/
│ │ │ └── dashboard.html # Responsive HTML5 dashboard & real-time telemetry
│ │ └── static/
│ │ ├── css/dashboard.css
│ │ └── js/dashboard.js
│ ├── hud/ # Always-On-Top Desktop HUD & Telemetry Overlay
│ │ ├── __init__.py
│ │ └── overlay.py # Tkinter floating dark obsidian window
│ ├── tray/ # Windows system tray controller
│ │ └── app.py # Pystray background service & menus
│ ├── mcp/ # Model Context Protocol endpoints
│ │ ├── __init__.py
│ │ └── server.py # Antigravity & Claude Desktop integration
├── scripts/ # Platform launchers & utilities
│ ├── start_cascade.bat # Windows batch launcher
│ ├── start_tray.vbs # Silent windowless tray runner
│ ├── start_hud.bat # Desktop HUD launcher
│ ├── start_hud.vbs # Silent windowless HUD launcher
│ ├── setup.bat / setup.sh # One-click installers
│ └── create_shortcuts.ps1 # Desktop & startup shortcut generator
├── tests/ # Comprehensive automated test suites
│ ├── test_gateway.py # End-to-end API integration tests
│ ├── test_resilience.py # Sub-5ms SLA, structural validation & lookahead tests
│ └── test_new_features.py # Loop breaker, IDE auto-config, dollar savings & /verify tests
├── assets/ # Icons & diagrams
├── config.yaml # Gateway configuration
├── pyproject.toml # Modern pip/uv packaging metadata
└── requirements.txt
config.yaml)server:
host: "127.0.0.1"
port: 8000
local:
provider: "ollama"
base_url: "http://127.0.0.1:11434"
# Set empty to let the hardware profiler auto-select the best model for your GPU
primary_model: ""
max_context_tokens: 32768
timeout_seconds: 120
cloud:
provider: "gemini"
default_model: "gemini-2.5-flash"
pro_model: "gemini-2.5-pro"
max_context_tokens: 1048576
biasing:
mode: "adaptive" # "adaptive", "manual", or "local_only"
bias_factor: 0.35 # Baseline local bias (0.0 to 1.0)
cloud_token_budget: 500000
allow_context_overflow_to_cloud: true
This project is licensed under the MIT License.
Hardware-adaptive local LLM & cloud cascading gateway. Sub-5ms intelligent routing, 3-token lookahead failover, asymmetric verification, 1-click IDE config, and MCP for $0 token cost.
เครื่องมือ
ยังไม่มีรายการเครื่องมือ เซิร์ฟเวอร์แบบรีโมตจะได้รายการหลังการตรวจสอบสถานะครั้งแรก
FAQ
คัดลอกคอนฟิก stdio สำหรับ Cursor, Claude Desktop, Claude Code หรือ VS Code ได้ในหน้านี้ คอนฟิกนี้ใช้การติดตั้งของผู้เผยแพร่ ไม่ใช่พร็อกซีของ PlugMCP
ระเบียนนี้ไม่ได้ระบุตัวแปรสภาพแวดล้อมที่จำเป็น
CascadeGateway ทำงานแบบโลคัลผ่าน stdio
ตรวจสอบล่าสุดเมื่อ 2026-10-28 Hardware-adaptive local LLM & cloud cascading gateway. Sub-5ms intelligent routing, 3-token lookahead failover, asymmetric verification, 1-click IDE config, and MCP for $0 token cost.