๐ง PdfToMem
Turn PDFs into structured, queryable memoryโbuilt for LLMs.
Large Language Models struggle with memory. PdfToMem makes it effortless.
By combining reasoning-powered ingestion, structured retrieval, and a multi-agent architecture, it transforms unstructured PDFs into rich memory representations.
It exposes a powerful MCP Server to coordinate ingestion, reasoning, storage, and queryingโusing state-of-the-art tools like LlamaIndex and LangGraph.
๐ฅ Video Walkthrough
๐ง MCP Server โ Memory Control Plane
The MCP Server is the brain of PdfToMem. It leverages tool-based reasoning to:
- ๐๏ธ Design ingestion pipelines
- ๐งฉ Choose the right memory representation
- ๐ Index and structure PDF content for retrieval
Use tools like determine_memory_architecture to automatically infer the optimal memory structure using LlamaIndex abstractions:
VectorIndexQueryEngineLLMSelector
๐ ๏ธ Tool-Driven Parsing โ Powered by LangGraph
PdfToMem employs modular tools to extract structured data from PDFs:
- ๐ Text Extraction
- ๐ Table Detection
- ๐งพ OCR for Scanned Docs
- ๐ผ๏ธ Image Extraction
- ๐ธ PDF Screenshots
These tools feed into the multi-agent system, enabling intelligent, context-aware processing.
๐ค Multi-Agent Architecture
PdfToMem uses a LangGraph-based orchestrator to coordinate specialized agents. Each agent performs a distinct task:
- ๐ต๏ธ Extractor Agent โ pulls raw content
- ๐ง Semantic Agent โ applies embedding & understanding
- โ๏ธ Segmenter Agent โ splits content intelligently
- ๐ธ๏ธ Relationship Agent โ builds semantic links
The orchestrator decides when and how to invoke these agents based on the content type and reasoning goals.
๐งญ Memory Planning โ Reasoning Over Representation
Once the data is structured, a planning model (currently o3-mini) determines the best memory format using LlamaIndex components:
- ๐งฑ
SimpleSentenceIndex - ๐
AutoMergingIndex - ๐งฎ Custom tool-enhanced indices
This reasoning-driven approach ensures optimal retrieval performance tailored to the document.
๐พ Storage โ Flexible, Dynamic, Queryable
Based on the selected memory representation, a FastAPI-powered Storage Service builds a tailored query engine for each PDF.
- ๐ Built-in search and retrieval
- ๐ง Vector and hybrid index support
- ๐งช Modular for experimentation
The storage is designed to support scalable memory systems across domains.
๐ฅ๏ธ MCP Client โ Full Control with a Friendly UI
The MCP Client is a React-based interface for controlling the full lifecycle:
- โ๏ธ Configure agents, tools, memory plans
- ๐ Upload and ingest PDFs
- ๐ Preview structured chunks
- ๐งช Query memory and view responses
Everything is interactive, inspectable, and customizable.
๐ Why PdfToMem?
- โ LLM-optimized ingestion & memory
- ๐งฉ Modular tools & agents
- ๐ง Reasoning-based memory planning
- ๐ฌ Queryable representations via LlamaIndex
- ๐ UI + API for full pipeline control
๐ ๏ธ Contributions welcome! Help us build the future of intelligent memory systems.
๐ MIT License
