Mnemosyne

Mnemosyne — Memory Distillation Engine

Mnemosyne — the Greek goddess of memory: memory never dies, is always reconstructable, and persists across sessions.

Compile any long-term interaction into an evolving human cognitive model: identity, preference, goal, emotion, relationship. Designed for companion AIs to maintain consistent persona across sessions.

Facts come from compilation, not guesswork. State comes from events, not prompts. Long-term cognition comes from the model, not context window length.

About the Name

Mnemosyne is the Greek goddess of memory — mother of the nine Muses, keeper of everything that must not be forgotten. The name is a promise made concrete by this engine:

• Memory never dies — facts are persisted in SQLite and survive across sessions, independent of the context window; memory_export/memory_import back them up and move them between machines intact.
• Reconstructable — every fact carries an evidence chain (EvidenceRef) and a scoring rule, so a persona's evolution timeline can be rebuilt from raw history at any time — nothing is guessed, everything is traceable.
• Continuity across sessions — cognition is in the model, not in the prompt: the store, not the context window, is the source of truth.

Just as Mnemosyne let poets and heroes remember, this engine lets a companion AI remember its user — deterministically, without an LLM in the loop.

Core Architecture

                Language Frontend
                     │
                     ▼
            Observation Compiler
                     │
                     ▼
            Knowledge Compiler
                     │
                     ▼
             Snapshot Builder
                     │
                     ▼
              Cognitive Context

Compilation Pipeline

Core Principles

• Facts from compilation: All Facts are extracted from raw text by the compiler with evidence chain (EvidenceRef) tracing — no LLM guessing involved.
• State from events: An Entity's current state is aggregated from its Fact timeline, not from ad-hoc prompt construction.
• Long-term cognition from the model: Cognition state persists in SQLite, evolves across sessions, independent of context window size.

MCP Tool Overview

Cognitive State Tools

Memory Distillation Tools

Knowledge Ingestion / External Source Tools

Mnemosyne Knowledge Query Tools

Companion Persona Tools

Persona-consistency tools for companion AIs (keeping the persona stable across sessions): inject a structured persona card, guard a draft reply against the stored persona, track the agent↔user relationship, and rebuild the persona's evolution timeline.

V1 Legacy Character Tools

portrait_extract (resume → person portrait) was removed in favor of the cognition-Facts conversation pipeline, which is the supported path for companion-AI persona profiling.

Semantic Similarity Without an LLM

The system is deliberately LLM-free for retrieval and persona reasoning: all semantic approximation is done with deterministic, offline algorithms, so the server has zero API-key dependency and its behavior is reproducible across runs. The techniques, in order of increasing semantic reach:

1. Keyword matching — FTS5

MEMORY_VECTOR_DIM=0 (the default) enables SQLite FTS5 keyword search with proper Unicode handling and escaped LIKE wildcards. Exact and substring matches are answered directly from the index — O(log n), no embedding needed.

2. Vector retrieval — cosine similarity

When MEMORY_VECTOR_DIM>0 and an embedding provider is configured (MEMORY_EMBEDDING_PROVIDER=openai|ollama, a remote API), text is embedded and queried by cosine similarity:

• brute_force.rs — exact O(N) full scan (the ground truth);
• hnsw.rs — approximate nearest neighbor for large graphs, with an agreed convention on degenerate inputs: a zero vector returns distance sqrt(2) (cosine = 0.0), identical to the brute-force scan, so both indexes agree.

3. Hybrid retrieval

MEMORY_RETRIEVAL_MODE=hybrid merges keyword hits and vector hits; results without a vector (e.g. legacy rows) fall back to keyword-only scoring, so the retrieval path never silently drops entries that lack an embedding.

4. Character-bigram Jaccard for names & aliases

For entity-name matching (compiler/resolver.rs) each name is embedded as its set of character bigrams (e.g. 张三 → {张三}; 张飞 → {张飞}), and similarity is the Jaccard overlap of the two bigram sets — no vector model required. This catches transliteration/copy errors and partial-name aliases deterministically.

5. Shared-bigram heuristics for persona consistency

persona_check (no-LLM path) compares a draft reply against stored persona facts with negation detection + shared-bigram overlap: two statements about the same bigram topic are flagged as conflict/drift without any model — 我讨厌应酬 vs 我喜欢安稳 are correctly not marked as a stance flip because they share fewer than 2 bigrams.

6. Rule-driven compilation

The whole extraction pipeline is rules, not inference: Aho-Corasick verb matching (observation_compiler), deterministic event/relation builders (compiler/extract.rs, timeline.rs), key-event importance scoring (evidence richness + participant centrality + turning-point flag), and rule-based relationship state updates (relationship_update) all run offline and are fully reproducible.

Net effect: every fact, edge, and retrieved result carries a traceable evidence chain or scoring rule — the "no LLM guessing" guarantee from Core Principles holds end to end.

IDE Integration (MCP)

The server speaks standard MCP over stdio (or HTTP+SSE), so it plugs into any IDE/agent that supports MCP clients — Claude Desktop, Cursor, VS Code, JetBrains, or a custom agent framework.

stdio (local, recommended for IDE)

// .mcp.json (VS Code / Cursor / Claude Desktop style)
{
  "mcpServers": {
    "mnemosyne": {
      "command": "cargo",
      "args": ["run", "--bin", "mnemosyne", "serve"],
      "cwd": "/abs/path/to/memory_distill"
    }
  }
}

Tip: cargo run compiles on first connect. For a snappy IDE experience build once (cargo build --release) and point command at target/release/mnemosyne with args: ["serve"].

HTTP+SSE (remote, requires token)

The HTTP transport exposes two endpoints: GET /sse (Server-Sent Events stream) and POST /message (JSON-RPC). Configure the MCP client with the SSE endpoint URL:

{
  "mcpServers": {
    "mnemosyne": {
      "url": "http://host:5609/sse",
      "headers": {
        "Authorization": "Bearer <your-token>",
        "x-mcp-session-id": "<stable-id-per-client>"
      }
    }
  }
}

HTTP serving refuses to start without --http-token (see Configuration); each concurrent client should send a stable x-mcp-session-id so its SSE stream only receives its own responses. For remote deployments use https:// — the bearer token would otherwise travel in cleartext.

What the AI can do once connected

• Distill the current conversation: the host passes the ongoing messages[] to memory_compile / memory_context_check and gets structured facts, decisions, session state and (optionally) long-term distilled memories — persisted to SQLite for cross-session continuity.
• Query the knowledge graph: inspect_entity, timeline, relation_graph, search_graph, trace_path, evidence.
• Guard persona consistency: persona_check / persona_inject / relationship_update keep a companion AI's persona from drifting.

Note: MCP tools are called by the host — the server never auto-reads the conversation. To auto-distill every turn, instruct the host (system prompt or post-turn hook) to invoke memory_compile with the current messages.

Installation

Option 1 — Install script (recommended)

scripts/install.sh detects your platform/architecture, downloads the matching release archive, and unpacks it into a .mnemosyne/ directory:

# macOS / Linux (bash)
curl -fsSL https://raw.githubusercontent.com/Timwood0x10/Mnemosyne/main/scripts/install.sh | bash

# Or clone and run it locally
git clone https://github.com/Timwood0x10/Mnemosyne.git
cd Mnemosyne
./scripts/install.sh          # latest release
./scripts/install.sh v0.1.2   # a specific version

What it produces — everything lives in one directory, so the binary always finds its resources next to itself:

~/.mnemosyne/
├── mnemosyne            # the binary (mnemosyne.exe on Windows)
├── markers_zh.json      # Chinese observation-marker word list (editable)
└── markers_en.json      # English observation-marker word list (editable)

Run it:

~/.mnemosyne/mnemosyne serve

Option 2 — Manual download

Download the archive for your platform from the Releases page:

# macOS / Linux
mkdir -p ~/.mnemosyne && tar -xzf mnemosyne-<platform>.tar.gz -C ~/.mnemosyne
chmod +x ~/.mnemosyne/mnemosyne
~/.mnemosyne/mnemosyne --version

# Windows: extract with your archive tool, then run mnemosyne.exe

Customizing the marker word lists

markers_zh.json and markers_en.json decide which words in a conversation produce facts. Each file maps an action (feel, plan, want, dislike, belief, stuck, life_event, …) to a list of trigger words. Edit them to:

• add your own vocabulary (modern slang, domain terms, personal quirks);
• remove words that cause false positives;
• tune what the engine extracts about the user.

No recompile needed — just edit the JSON and restart. If the files are absent the binary falls back to built-in defaults.

Option 3 — Build from source

Requires a Rust toolchain (see rust-toolchain / Cargo.toml for the MSRV).

git clone https://github.com/Timwood0x10/Mnemosyne.git
cd Mnemosyne
cargo build --release
./target/release/mnemosyne --version

Then continue with Quick Start.

Quick Start

# Zero config: SQLite only, no API key needed
cargo run --bin mnemosyne \
  --embedding-provider none \
  --retrieval-mode keyword \
  --db-path ./knowledge.db

CLI Commands

Testing

make check      # cargo clippy + cargo check (0 error, 0 warning)
make test       # 700+ unit + integration tests (nextest, ~18s)

# Full Romance of the Three Kingdoms compilation test
cargo test --test sanguo_compile e2e_sanguo -- --nocapture

Test Corpora

All test corpora live in corpus/ (text, PDF and dialog-JSON), with per-novel entity profile packs in config/entity_profiles/.

Entity profile packs (config/entity_profiles/): sanguo.json, shuihu.json, honglou.json, xiyou.json, fengshen.json, warandpeace.json — each maps the novel's canonical entity names/aliases for the compiler's dictionary (JsonEntityProvider).

Slow full-corpus runs (e.g. the 7-novel generalize_corpus_regression) are #[ignore]d by default; run them explicitly with --ignored.

Configuration

Development

make check      # cargo clippy + cargo check
make test       # All 320+ tests
make fmt        # Format code

Module Documentation

License

Apache-2.0