A Model Context Protocol (MCP) server for indexing and searching local document collections. Supports DOCX, HWPX, PDF (with OCR), Excel, PowerPoint, and audio (Whisper transcription, Windows-only), with hybrid keyword + semantic search optimized for Korean and English.
MCP RAG server with hybrid search, multi-KB support, and AI-powered chunk contextualization.
fieldcure-mcp-rag
├── serve --base-path <path> # Multi-KB MCP search server (stdio)
├── exec --path <kb-path> [--force] [--partial ...] # Headless indexing for a single KB
├── exec-queue --queue-file <path> [--sweep-all] # Process deferred indexing queue
├── prune-orphans --base-path <path> # Delete orphan KB folders
└── smoke-ocr --pdf <scanned.pdf> # Self-test: OCR a scanned PDF (Windows)
serve — read-only MCP server serving all knowledge bases under the base path. Single process handles multiple KBs via kb_id parameter. Can run while exec is indexing (SQLite WAL).
exec — scans source folders, chunks documents, contextualizes with AI, embeds, stores in SQLite. --partial re-runs only downstream stages when models change, preserving OCR output.
exec-queue — sequential orchestrator consuming a deferred indexing queue. One entry at a time, no GPU contention. --sweep-all processes deferred entries too (used at app shutdown).
prune-orphans — deletes orphan KB folders (GUID-named, no config.json). Protected folders (., _ prefix, -backup-) are never touched.
smoke-ocr — diagnostic mode. Loads a scanned PDF through the OCR fallback parser, prints recognized text to stdout, and exits 0 on a non-empty result. Surfaces DllNotFoundException / BadImageFormatException distinctly so a missing or arch-mismatched native is immediately visible. Useful for verifying that the OCR native path is wired correctly on a given host (notably win-arm64 dnx installs).
Features
Search
Hybrid BM25 + vector search with Reciprocal Rank Fusion (RRF)
BM25-only fallback when no embedding provider is configured
SIMD-accelerated cosine similarity via System.Numerics.Vector
FTS5 trigram index for substring and CJK-friendly keyword matching
Indexing
Incremental indexing with SHA256 change detection
AI-powered chunk contextualization with bilingual keyword enrichment (see Chunk Contextualization)
2-commit pipeline preserves expensive upstream work across embedding failures (see How Indexing Works)
Math equation extraction from DOCX/HWPX as [math: LaTeX] blocks
PDF with OCR fallback (Tesseract eng+kor) for scanned pages
Audio transcription (.mp3, .wav, .m4a, .ogg, .flac, .webm) via Whisper.net — Windows-only. Model size (Tiny→Large) is auto-selected from detected GPU/RAM/cores at startup; each transcript chunk records audio.model_size and audio.transcribed_at for future reindex auditing
Cross-process indexing lock with stale PID auto-cleanup
Orphan cleanup for deleted files
Queue Orchestrator
All indexing requests flow through start_reindex MCP tool — no direct exec spawn
Scope merge rules: full ⊃ contextualization ⊃ embedding (duplicate requests upgrade, not duplicate)
PID-based orchestrator lock with reuse defense (orchestrator.lock)
Deferred indexing for app-shutdown batch processing (--sweep-all)
Operations
Multi-KB serve: single process serves all knowledge bases under a base path, lazy-loaded per KB
SQLite WAL mode allows search during indexing
Graceful shutdown via cancel file
Per-KB config.json with provider configuration
Integration
Ollama native — embedding via /api/embed, contextualization via /api/chat with keep_alive and num_ctx support. Requires Ollama 0.4.0+.
OpenAI-compatible — embedding via /v1/embeddings, contextualization via /v1/chat/completions. Works with OpenAI, Azure OpenAI, Groq, LM Studio, Together AI.
Gemini native — embedding via /v1beta/models/{model}:embedContent with task_type asymmetric retrieval (RETRIEVAL_DOCUMENT / RETRIEVAL_QUERY) and Matryoshka dimension truncation (768 / 1536 / 3072). gemini-embedding-2, multilingual, 8k token input.
Anthropic — contextualization via /v1/messages.
API keys via environment variables — OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. Batch indexing commands (exec, exec-queue) are env-var-only. Interactive MCP search can fall back to MCP elicitation when the client supports it.
Standard MCP stdio transport (JSON-RPC over stdin/stdout)
Chunk Contextualization
Standard RAG chunking loses context — a sentence about "the protocol" becomes ambiguous when ripped from its surrounding paragraphs. This server addresses that with Unified Chunk Contextualization: a single LLM call per chunk that produces both contextual framing and bilingual (Korean + English) keywords in one pass.
The result is stored alongside the original chunk text:
Original text is preserved for accurate retrieval display
Contextualized text is what gets embedded and indexed in BM25
Bilingual keywords enable cross-lingual search — a Korean query can retrieve English documents and vice versa
This is enabled by setting contextualizer in config.json. It can be disabled (set provider/model to empty) if you prefer raw chunk indexing.
How Indexing Works
The exec command runs a 5-stage pipeline per file:
Extract — text from document (DOCX, PDF OCR, audio transcription, etc.)
Chunk — split into ~1000 char windows
Contextualize — LLM enrichment (optional, see above)
Embed — vector embedding via API
Persist — save to SQLite
For large files, Stage 1 alone can take 20+ minutes — OCR on a 596-page scanned PDF, or Whisper transcription of a multi-hour audio recording. The first audio file in any KB also pays a one-time ggml model download (cached under {UserProfile}/.fieldcure/whisper-models/). To prevent expensive upstream work from being lost when later stages fail, the pipeline uses a 2-commit model:
Why this matters: A 25-minute OCR result is persisted on disk before any embedding API call. If Stage 4 fails (network error, rate limit, token limit, process crash, even power loss), the chunks survive. The next exec hash-skips the file (no OCR re-run) and the deferred retry pass attempts only Stage 4.
Per-Chunk Failure Isolation (Binary Split)
If a single chunk in a file exceeds the embedding model's token limit (e.g., a math-dense page in a textbook), the binary split algorithm isolates that one chunk:
Result: 1249 chunks indexed, only chunk 846 marked Failed. The file's status becomes Degraded — partially searchable instead of completely missing.
Deferred Retry Pass
Each exec ends with a retry pass over any chunks left in PendingEmbedding state from previous runs:
Reads enriched text from DB — no OCR or contextualization re-run
Calls the embedding API only — typically seconds, not minutes
Up to 3 retries per chunk; on exhaustion, the chunk is marked Failed
Auth errors (401/403) flag the provider as unavailable and skip the rest of the pass
File States
Status
Meaning
Hash-skip behavior
Ready
Fully indexed
Skip if hash matches
Degraded
Some chunks failed (binary-split isolated)
Skip if hash matches
PartiallyDeferred
Chunks pending embedding retry
Main loop skips; deferred pass picks up
Failed
Extraction or repeated embedding failure
Skip; requires --force to retry
NeedsAction
User intervention required
Skip with separate counter
Schema Versioning
Each KB DB carries a PRAGMA user_version tag. The exec command migrates older schemas automatically as part of InitializeSchema(). The serve command opens DBs read-only and never triggers migration — older-schema KBs continue to serve search queries correctly while their new-feature columns remain unused.
Installation
dotnet tool (recommended)
dotnet tool install -g FieldCure.Mcp.Rag
From source
git clone https://github.com/fieldcure/fieldcure-mcp-rag.git
cd fieldcure-mcp-rag
dotnet build
OCR: Windows x64 only — Tesseract OCR for scanned PDFs loads lazily on first use (Windows only). On other platforms, PDFs with embedded text work normally; scanned pages without a text layer are silently skipped.
An embedding provider (Ollama, OpenAI, etc.) — optional, BM25 search works without it
Ollama 0.4.0 or later (if using Ollama for embedding or contextualization)
Quick Start
Index a folder and search it without any embedding setup (BM25 only):
In serve mode, search_documents can also prompt via MCP elicitation when the client supports it. In exec and exec-queue, missing keys must be provided via environment variables.