LLMC: Large Language Model Compressor
Stop burning money on LLM tokens. Get 70-95% cost reduction through local RAG, intelligent routing, and containerized security.
⚡ What is LLMC?
LLMC is a local-first RAG (Retrieval Augmented Generation) engine and intelligent router designed to drastically reduce the cost of using Large Language Models with your codebase.
Instead of sending your entire codebase to Claude or GPT-4, LLMC indexes your code locally, finds the exact relevant snippets (functions, classes, docs), and sends only what matters.
graph LR
A[User Query] --> B(LLMC Router);
B --> C{Local Index};
C -->|Search| D[Relevant Context];
D -->|Trim & Pack| E[Optimized Prompt];
E --> F[LLM API];
F --> G[Answer];
style B fill:#f9f,stroke:#333,stroke-width:2px
style E fill:#bfb,stroke:#333,stroke-width:2px
🚀 Quick Start
Get up and running in seconds.
1. Install
# One-line install
curl -sSL https://raw.githubusercontent.com/vmlinuzx/llmc/main/install.sh | bash
# Or via pip
pip install "git+https://github.com/vmlinuzx/llmc.git#egg=llmcwrapper[rag,tui,agent]"
2. Index Your Code
cd /path/to/your/project
llmc-cli repo register .
3. Save Money
# Search without using ANY tokens
llmc-cli search "authentication middleware"
# Launch the visual dashboard
llmc-cli tui
✨ Key Features
🔍 Deep Dive
- Local SQLite Index: Stores text + metadata without external dependencies.
- Smart Embeddings: Caches embeddings to avoid re-computing unchanged files.
- Context Trimmer: Packs the most relevant spans into a fixed token budget.
- Enrichment: Uses small local models to tag and summarize code for better retrieval.
- Hybrid Mode: Trusted clients get direct host access (~76% cheaper than docker overhead).
- Container Isolation: Untrusted inputs run in Docker/nsjail.
- Defense in Depth: Even if an LLM is "jailbroken" by prompt injection, it can't escape the container.
- Intelligent Failover: Cascades from Local → Cheap Cloud → Premium Models.
- Cost Tracking: Hard budget caps to prevent surprise bills.
- Rate Limiting: Automatic token bucket throttling for API providers.
📚 Documentation
Full documentation is available in the DOCS/ directory:
- Getting Started — Installation and quickstart
- User Guide — Configuration and daily usage
- Operations — Running the daemon and MCP integration
- Architecture — System design and internals
- Reference — CLI, config, and MCP tool reference
📜 History
Originally created by David Carroll (the worst paragliding pilot in the TX Panhandle) after burning through his weekly API limits in days. This tool was born from the necessity to code more while spending less.
🤝 Contributing
We welcome PRs! Please check CONTRIBUTING.md before starting.
- Fork the repo
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
Current Release: v0.9.1 "Back From Vacation"
