lore
A local knowledge base you can search from the terminal or serve to agents over MCP. lore ingests your documents -- local files, websites, git repos, feeds, S3, YouTube, email, shell commands, upstream MCP servers, 90+ formats -- and indexes everything with full-text search. No external services, no infrastructure to manage. One binary, one config, one store -- or as many as you need.
Think man pages for your projects -- for humans and agents alike.
Install
Install script (Linux and macOS):
curl -fsSL https://github.com/timorunge/lore/releases/latest/download/lore-cli-installer.sh | bash
Homebrew (macOS and Linux):
brew install timorunge/tap/lore-cli
Or download a pre-built binary directly from the releases page.
Windows: Pre-built binaries are available for Linux and macOS. On Windows, use WSL (recommended) or build from source with
cargo install(OCR requires cmake and is not supported on MSVC -- omit with--no-default-features --features ingest,mcp).
Build from source:
git clone https://github.com/timorunge/lore.git
cd lore
cargo install --locked --path cli
Optional feature flags:
cargo install --locked --path cli --features llm # with LLM enrichment
cargo install --locked --path cli --features llm,s3 # with LLM + S3 support
cargo install --locked --path cli --all-features # all optional features
cargo install --locked --path cli --no-default-features # minimal build (no OCR, no MCP)
# Windows (native, no OCR):
cargo install --locked --path cli --no-default-features --features ingest,mcp
Quick start
cd my-project
lore init # generates .lore/lore.yaml from your project
lore ingest # build the store
lore search "authentication" # search from the terminal
lore serve # or serve to agents over MCP
lore init scans your directory, detects documentation folders and
README files, and generates a config with pre-filled sources. Review it,
ingest, and search -- from your terminal or through any MCP client.
Try it without your own docs
If you cloned the repo, the gutenberg example works out of the box:
cd examples/gutenberg
lore ingest
lore search "love and death"
Without a checkout, fetch the config directly:
mkdir tryout && cd tryout && mkdir -p .lore
curl -fsSL https://raw.githubusercontent.com/timorunge/lore/main/examples/gutenberg/lore.yaml \
-o .lore/lore.yaml
lore ingest
lore search "love and death"
This fetches classic novels from Project Gutenberg and indexes them -- no local documents required. See examples/ for more ready-to-run configs.
Use cases
Give your AI assistant project context -- index your READMEs, design docs, ADRs, and runbooks. Connect via MCP. Your AI now knows your project without stuffing everything into the prompt.
sources:
- path: docs/
topic: Internal
Offline documentation search -- index vendor docs, API references, or internal wikis. Search from the terminal with no browser and no internet.
sources:
- sitemap: https://docs.example.com/sitemap.xml
topic: API Reference
Team knowledge base -- point lore at a shared git repo of documents.
Everyone runs lore ingest locally. No server, no SaaS, no vendor
lock-in.
sources:
- git: https://github.com/org/team-docs
glob: "**/*.md"
topic: Team
Federated search -- keep knowledge bases separate but query them together. Each config has its own store and update schedule; lore merges results at query time.
lore search "auth" -c project.yaml -c vendor.yaml
MCP integration
Serve your knowledge base to AI assistants over the Model Context Protocol.
Stdio (Claude Code, Kiro, Cursor -- any MCP client that launches subprocesses):
{
"mcpServers": {
"my-docs": {
"command": "lore",
"args": ["serve"],
"env": {
"LORE_CONFIG": "/path/to/.lore/lore.yaml"
}
}
}
}
Streamable HTTP (web apps, remote agents, multiple clients):
lore serve --transport http --port 8080
{
"mcpServers": {
"my-docs": {
"url": "http://localhost:8080/mcp"
}
}
}
Six read-only tools: lore_info, lore_list_topics, lore_search,
lore_read_topic, lore_list_docs, and lore_read_doc.
MCP resources for clients that support browsing: lore://info,
plus one lore://topics/{name} and one lore://docs/{source} resource
per topic and document. All discoverable via list_resources.
Serve multiple knowledge bases through a single MCP server -- pass
multiple -c flags and lore federates them at query time:
lore serve -c project.yaml -c vendor.yaml
/lore skill -- lore ships a /lore skill in
skills/lore/ for AI coding assistants that support
skills. Type /lore for interactive help with setup, config authoring,
and troubleshooting.
See MCP Integration for transport options, multi-KB setups, and the full tool reference.
Configuration
Each knowledge base is defined by a YAML config file. Mix source types and assign topics for independent search:
name: my-knowledge-base
base_dir: ..
sources:
# All source keys accept a single string or a list of strings.
- path: ./docs
glob: "**/*.md"
topic: Internal
- git: https://github.com/org/public-docs
glob: "**/*.md"
topic: Public Docs
- sitemap: https://docs.example.com/sitemap.xml
include: "/reference/"
topic: API Reference
- youtube: https://www.youtube.com/watch?v=JZfJTSlhOXM
lang: en
topic: Talks
The YouTube source spawns yt-dlp
as a subprocess to fetch transcripts -- yt-dlp must be on PATH.
Playlists and channels are also supported.
URL-based sources support custom HTTP headers for authenticated
endpoints. Header values support ${LORE_*} environment variable
expansion (only variables with the LORE_ prefix are expanded):
- sitemap: https://internal.corp/sitemap.xml
headers:
Authorization: "Bearer ${LORE_DOCS_TOKEN}"
Processing profiles let you tune chunking and metadata extraction per source. Define named presets and reference them inline, or override inline directly:
base_dir: ..
sources:
- path: docs/ # uses global defaults
- path: src/
processing: code # uses code preset
- path: special/
processing: # inline override
max_chunk_chars: 3000
processing:
max_chunk_chars: 1600
presets:
code:
extract: none
max_chunk_chars: 800
See the examples/ directory for complete, runnable configs and the Configuration Reference for all options.
How it works
sources lore consumers
| | |
| local, URL, git, | |
| sitemap, feed, S3, | |
| YouTube, maildir, | |
| exec, MCP | |
|----------------------->| |
| | extract text (kreuzberg) |
| | extract metadata |
| | apply pipeline transforms |
| | chunk by structure |
| | index (tantivy) |
| | |
| | lore search (terminal) |
| |<----- you ----- or --------->|
| | lore serve (stdio / http) |
| |<-----------------------------|
| | search / browse / retrieve |
| |----------------------------->|
Documents flow left to right during lore ingest. At query time, you
search from the terminal with lore search, or agents query through
six read-only MCP tools via lore serve. The store lives in
.lore/store by default (configurable via store.path or
LORE_STORE_PATH) -- no external services.
What lore does
- 10 source types -- local files, URLs, git repos, sitemaps, RSS/Atom feeds, S3, YouTube (via yt-dlp), maildir, shell commands, and upstream MCP servers
- 90+ document formats -- PDF, DOCX, XLSX, HTML, email, EPUB, archives, Markdown, Org-mode, LaTeX, source code, and more via kreuzberg
- Metadata extraction -- title, author, language, created date,
topic, and tags from binary metadata, frontmatter, Org-mode headers,
and content heuristics; choose
auto,builtin,kreuzberg, ornoneper source - Smart chunking -- respects document structure (headings, sections); configurable size limits
- Processing profiles -- per-source presets and inline overrides for chunk size, metadata extraction, and custom transform pipelines
- Incremental updates -- only new or modified documents are re-processed; periodic commits let you resume cancelled ingests
- Embedded index -- full-text search via Tantivy, stored locally, no external database
Search from the terminal
Query your knowledge base without leaving the terminal:
lore search "authentication" # free-text search
lore search "auth" --topic Security # filter by topic
lore topics # list all topics
lore docs # list all documents
lore read docs/auth.md # read a document
lore read docs/auth.md --full # read as continuous text
lore status # show what changed since last ingest
lore completions zsh # generate shell completions
Query and listing commands support --json for piping to jq and other tools.
Pass multiple -c flags for federated search across knowledge bases. See
CLI Reference for the full flag reference.
Documentation
Data responsibility
You are responsible for ensuring you have the right to ingest, index, and serve the content you configure. When LLM enrichment is enabled, document content is sent to the configured provider -- for sensitive data, consider a local provider like Ollama. lore does not phone home or collect telemetry. See Security for details.
License
MIT -- see LICENSE.
Every tool needs a mass-appealing origin story these days, so here is ours.
lore is an old lady with tentacles. She hoards knowledge like a kraken hoards
shipwrecks -- reaching into your PDFs, your contracts, your dusty email
archives, pulling out what matters, and indexing it before you even knew you
needed it. She reads everything you give her -- twelve nines of
reliability, one nine more than Amazon promises for not losing your
files. She does not know what downtime means. She claims she was here
before the word was invented -- her git log disagrees. She is part librarian,
part eldritch horror, part folklore -- an oral tradition for the digital
age, except she actually remembers things correctly -- until you
lore ingest --recreate and wipe her memory clean. She does not hold
grudges. She just re-reads everything -- and fast. You will see progress
bars flicker and vanish before you can read their labels, documents
streaming past like debris in a whirlpool. By the time you glance back
at the terminal, she is already done and waiting for your next question.
Old lady with tentacles, remember? Parallel I/O.
She is not a fancy AI. She does not hallucinate your documents. She was, however, largely written by one -- but that is a family secret. She uses BM25 like it is 1994 because it works and because she does not need a GPU to tell you where you wrote down that one authentication thing three months ago -- and to quietly surface a memo you never opened that would have saved you the trouble. She is a single binary -- all tentacles in one body, hunting alone in the deep. No cloud above her, the sun does not reach that far down. No dependencies, no entourage, no committee. Just her and strong opinions about how to slice a shipwreck into readable pieces.
Will she change your life? No. If your problem is 200 Markdown files, use ripgrep -- you are fine. But when it is PDFs, Word documents, scanned invoices, HTML exports, and that one spreadsheet Karen from accounting sent as a .zip and you are too tired to care which tool opens what -- that is where lore lives. She will find that stupid Q3 number at 11 PM so you do not have to open Excel. As for Karen -- lore says she was "processed." We do not ask follow-up questions.
Some say she keeps more than one lair. One for the things you wrote, one for the things sent to you, one for the things you inherited and never dared open -- each hoard sealed off, each minding its own business. They do not gossip. But she remembers every single one, and if you ask the right question she will search them all at once -- one question, every lair, answers rising from the deep already sorted by which ones matter most. She calls it federation. We call it unsettling.
Give her your documents. She loves reading them. She loves reading them to you even more -- come in, sit down, have a cookie -- yes, granny's cookies. She baked them. Do not ask what is in them.
You already found her.
She is still reading.
