This repo builds a production-grade agentic RAG system from scratch. A single-file RAG script evolves, milestone by milestone, into a secured multi-agent platform with a real MCP server, enterprise middleware, and automated evaluation gates. Every milestone ships with the actual bugs hit along the way and how they got fixed, not just a working final version.
Built with: LangGraph · ChromaDB · Model Context Protocol (MCP) · Langfuse · RAGAS · Gemini API
Table of Contents
- What You Will Build
- Learning Path & Roadmap
- Who This Is For
- Study Guides, Glossaries & Notes
- Exercises, Quizzes, and Runbooks
- Repository Structure
- Environment Setup
- Real Production Bugs We Hit (And Fixed)
- Contributions Welcome
What You Will Build (Projects)
Instead of just reading theory, you build a complete, production-grade knowledge base agent, piece by piece:
- A Hybrid RAG Pipeline: ingest PDFs, chunk them, embed them, and query with Hybrid Search (BM25 keyword + vector semantic search) over ChromaDB, with cross-encoder re-ranking.
- A LangGraph AI Agent: an orchestrator that routes queries, calls tools, and holds short-term conversational memory as an explicit state machine.
- An Agentic Memory System: an inverted pipeline that extracts durable facts from conversation and recalls them across sessions.
- A Multi-Agent Architecture: an Orchestrator-Worker pattern that securely routes tasks between specialized sub-agents (document search vs. memory).
- A Real MCP Server & Enterprise Gateway: expose your tools over the Model Context Protocol behind a secured API gateway with auth, scoping, and multi-tenant isolation.
- Production Resilience Patterns: query rewriting, parallel tool execution, human-in-the-loop approval gates, graceful degradation, and an automated regression harness.
- Enterprise Middleware & Security: a request/response interceptor chain (auth, rate limiting, PII redaction), prompt-injection defenses for RAG, and enterprise connectors (Slack/Jira-style).
- Evaluation, Safety & Fine-Tuning: a real RAGAS eval harness, adversarial red-teaming, and hands-on LoRA fine-tuning.
- Scale & Multi-Tenancy Capstone: true tenant isolation, vector DB performance benchmarking at 100K+ chunks, and a formal system design write-up.
Learning Path & Roadmap
The project is structured into 10+ milestones:
How to Follow Along Hands-On (Time Travel)
The best way to understand this repository is to travel through its history! Every major milestone was developed incrementally, and the exact state of the codebase at the end of each milestone is preserved using Git tags.
If you want to see exactly what the project looked like when we finished the very first milestone, you can simply run:
git checkout milestone-1
You can read the code, run the specific exercises for that milestone, and then jump forward to git checkout milestone-2 to see exactly what changed.
The available tags line up with the milestones (e.g., milestone-1 through milestone-10). If you prefer to see the granular, step-by-step changes, you can also explore the Git commit history directly!
Who This Is For
- Backend/software engineers who already know how to code and want real system-design depth in AI, not just prompt-writing.
- Anyone prepping for Staff Engineer / Agentic Platform / AI Architect-style interviews who wants defensible, hands-on production stories instead of tutorial-only experience.
- Developers evaluating LangGraph vs. plain function-calling, or trying to understand what MCP is actually for beyond the spec docs.
Study Guides, Glossaries & Notes
Every milestone is paired with theory notes and a running, tiered terminology glossary, written plainly, with real-world analogies instead of textbook definitions.
👉 Start here: Master Notes Index 👉 Terminology Glossary
Exercises, Quizzes, and Runbooks
The best way to learn is by doing. Every milestone ships with explicit terminal commands and expected outputs so you can verify your own implementation.
Check out TESTING.md to run the local multi-agent system, exercise the RAG pipeline, and reproduce (then fix) real failure modes yourself.
Repository Structure
.
├── .env # Secrets and API Keys (Gemini & Langfuse)
├── doc_configs/ # Metadata registries (DocConfigs) for documents
├── docs/ # Folder to store PDFs and text files
├── src/ # Source code package
│ ├── agents/ # LangGraph Agents (Coordinator, Document, Memory)
│ ├── memory/ # Long-term conversational memory
│ ├── middleware/ # Auth, rate limiting, PII redaction, guardrails
│ ├── connectors/ # Enterprise connectors (Slack, Jira, etc.)
│ ├── eval/ # RAGAS evaluation harness and pre-deploy gates
│ ├── rag/ # Ingestion, embeddings, and parsers
│ └── tools/ # Custom agent tools
├── tests/ # Unit and integration tests
├── notes/ # Dedicated study guides, diagrams, and glossaries
├── main.py # Single CLI entry point to run all components
├── TESTING.md # Exercises and comprehensive runbooks
├── CONTRIBUTING.md # Guidelines for contributing to the repository
└── requirements.txt # Python dependency list
Environment Setup
1. Prerequisites
Ensure you have Python 3.12 installed on your system. Verify with:
python3.12 --version
2. Set up Virtual Environment
Create and activate the virtual environment:
python3.12 -m venv venv
source venv/bin/activate
pip install --upgrade pip
3. Install Dependencies
Install all required packages from the requirements.txt file:
pip install -r requirements.txt
4. Configuration
Create a .env file in the root directory:
GOOGLE_API_KEY=your_gemini_api_key_here
LANGFUSE_PUBLIC_KEY=your_langfuse_public_key_here
LANGFUSE_SECRET_KEY=your_langfuse_secret_key_here
LANGFUSE_HOST=https://cloud.langfuse.com
Quick Start Guide
Once your environment is set up and your API keys are configured, follow these 4 steps to run the agent from scratch without diving into the underlying code:
0. Configure Your Documents
- The Documents: Drop your PDF, TXT, or MD files into the
docs/folder located at the root of the project (or any other folder on your machine). - The Path: Open your
.envfile and explicitly tell the scripts where to look:
(Note: You can changeDEFAULT_KNOWLEDGE_BASE_PATH=./docs./docsto any absolute or relative path on your machine where your documents live). - The Doc Config: Open the
doc_configs/doc_configs.jsonfile. This tells the AI how to digest each specific document. Just add a new block for your exact filename to define how large the text chunks should be and whether to extract images (e.g.,captionorskip):{ "my_document.pdf": { "chunk_size": 1000, "chunk_overlap": 200, "image_strategy": "caption", "priority": 1 } }
1. Ingest into the Vector Database
Run this command to parse your documents, break them into chunks based on the config above, and save them into the local vector database:
python main.py ingest
2. (Optional) Load Mock Enterprise Connectors
Memory itself doesn't need to be pre-loaded: the agent dynamically builds its short-term and long-term memory in real-time as you chat!
However, this repository also teaches the "Enterprise Connectors Pattern". If you want to see how an agent handles external corporate data, run this command to load a set of simulated, fake Jira tickets and Slack messages into the database. (Note: As of the currently completed milestones, we do not provide real Slack/Jira integrations; this is purely a mock implementation to teach the architectural pattern without requiring corporate API keys).
python -m src.rag.ingest_connectors
3. Start Chatting!
Once ingestion is done, run this command to launch the fully-equipped agentic interface (which includes the intelligent router, sub-agents, security guardrails, and memory):
python main.py coordinator
You will immediately get an interactive prompt where you can start asking complex questions about your documents!
Real Production Bugs We Hit (And Fixed)
Not textbook exercises; these are actual failures diagnosed and fixed during development:
(→ all 10, with full write-ups, in the glossary and milestone notes)
Contributions Welcome
Found a typo in the notes? Want to add a new quiz, exercise, or a better architectural diagram? Contributions of all skill levels are welcome; see CONTRIBUTING.md for "Good First Issues".
If this repo helped you understand agentic AI system design, consider starring it, as it helps more people find it.