Synaptic Memory

Synaptic Memory

Default path: zero API calls at index time. Zero infra. Zero lock-in. A knowledge graph + MCP tool server for LLM agents, with hybrid retrieval, CDC-based live database sync, and Korean FTS built in.

한국어 README

5-minute start

pip install "synaptic-memory[sqlite,korean,vector]"
synaptic-quickstart --db quickstart.db

That command builds a tiny SQLite-backed graph and runs three searches — all without calling any LLM at indexing time. Omit --db for an in-memory, zero-dependency smoke test. Full source for the expanded example: examples/quickstart.py.

Why not just RAG?

Plain RAG usually answers from independent chunks. Synaptic builds a graph first, so an agent can search, follow relations, inspect structured rows, and remember which evidence helped.

It is not a vector database replacement. It is the graph and tool layer around your existing documents, SQL data, embedding endpoint, and agent runtime.

Build and search

import asyncio
from synaptic import SynapticGraph

async def main():
    # Any data → knowledge graph (CSV, JSONL, directory)
    graph = await SynapticGraph.from_data("./my_data/", preset="rag")
    try:
        result = await graph.search("my question")
        print(result.nodes[0].node.title if result.nodes else "no result")
    finally:
        await graph.close()

asyncio.run(main())

That's it. Auto-detects file format or DB schema, generates an ontology profile, ingests, indexes, builds FK edges.

Presets keep the common knobs compact:

# local: deterministic, no external services (default)
graph = await SynapticGraph.from_chunks(chunks, preset="local")

# rag: reads SYNAPTIC_EMBED_URL / SYNAPTIC_RERANK_URL if set
graph = await SynapticGraph.from_data("./docs/", preset="rag")

# agent: rag + deterministic component bridging for multi-turn exploration
graph = await SynapticGraph.from_data("./docs/", preset="agent")

Live database sync (CDC) — mode="cdc" enables incremental updates: tables with an updated_at-style column are read with a watermark filter, others fall back to per-row content hashing. Deletes are detected via a TEMP TABLE LEFT JOIN; FK rewires re-link the corresponding RELATED edges. Search results are identical to a full reload (locked in by a regression test). Supports SQLite, PostgreSQL, MySQL/MariaDB.

Office files (PDF/DOCX/PPTX/XLSX/HWP) are supported through the optional xgen-doc2chunk package. Install with pip install synaptic-memory[docs] or use from_chunks() with your own parser.

What it does

Your data (CSV, JSONL, PDF/DOCX/PPTX/XLSX/HWP, SQL database)
  ↓  auto-detect format / auto-discover DB schema + FKs
  ↓  DocumentIngester (text) / TableIngester / DbIngester
  ↓
Knowledge Graph
  ├─ Documents: Category → Document → Chunk
  └─ Structured: table rows as ENTITY nodes + RELATED edges (FKs)
  ↓
MCP tools → LLM agent explores via graph-aware multi-turn tool use

Two jobs, nothing else:

1. Build the graph well — cheap deterministic extraction by default
2. Give the LLM good tools — the agent decides what to search

Install

# Recommended local graph + MCP setup
pip install "synaptic-memory[sqlite,korean,vector,mcp]"

# Team / production graph on PostgreSQL + pgvector
pip install "synaptic-memory[postgresql,embedding,reranker]"

# Scale-out helpers: Kuzu graph + Qdrant vector + MinIO blob storage
pip install "synaptic-memory[scale]"

# Add this for the LangChain retriever example
pip install "synaptic-memory[langchain]"

# Or everything, including Postgres / Kuzu / Qdrant / MinIO
pip install "synaptic-memory[all]"

pip install synaptic-memory                # Core (zero deps, in-memory only)
pip install synaptic-memory[sqlite]        # + SQLite FTS5 backend
pip install synaptic-memory[korean]        # + Kiwi morphological analyzer
pip install synaptic-memory[vector]        # + usearch HNSW index
pip install synaptic-memory[mcp]           # + MCP server for Claude
pip install synaptic-memory[embedding]     # + aiohttp for embedding APIs
pip install synaptic-memory[reranker]      # + flashrank cross-encoder
pip install synaptic-memory[langchain]     # + LangChain retriever adapter
pip install synaptic-memory[postgresql]    # + asyncpg + pgvector
pip install synaptic-memory[mysql]         # + aiomysql DB ingest
pip install synaptic-memory[oracle]        # + oracledb DB ingest
pip install synaptic-memory[mssql]         # + aioodbc DB ingest
pip install synaptic-memory[kuzu]          # + embedded property graph backend
pip install synaptic-memory[qdrant]        # + Qdrant vector helper
pip install synaptic-memory[minio]         # + MinIO/S3-compatible blob helper
pip install synaptic-memory[scale]         # + Kuzu + Qdrant + MinIO + aiohttp
pip install synaptic-memory[docs]          # + xgen-doc2chunk (PDF/DOCX/PPTX/XLSX/HWP)

Infrastructure Integration

The default one-liner creates a local SQLite graph. For existing infrastructure, create the backend yourself, connect it, and pass it to from_data(), from_chunks(), or from_database().

from synaptic import SynapticGraph
from synaptic.backends.postgresql import PostgreSQLBackend

backend = PostgreSQLBackend("postgresql://user:pass@host:5432/synaptic")
await backend.connect()

graph = await SynapticGraph.from_data("./docs/", backend=backend, preset="rag")

Current backend roles:

Qdrant and MinIO are helper services, not full graph stores. Use them through CompositeBackend: graph storage keeps nodes/edges, Qdrant handles ANN vector search, and MinIO/S3-compatible storage offloads large Node.content.

from synaptic.backends.composite import CompositeBackend
from synaptic.backends.kuzu import KuzuBackend
from synaptic.backends.minio_store import MinIOBackend
from synaptic.backends.qdrant import QdrantBackend

backend = CompositeBackend(
    KuzuBackend("synaptic.kuzu"),
    vector=QdrantBackend("http://localhost:6333", collection="synaptic"),
    blob=MinIOBackend("localhost:9000", bucket="synaptic"),
)
await backend.connect()

graph = await SynapticGraph.from_data("./docs/", backend=backend, preset="scale")

The library gives you the backend contracts and the retrieval layer. For multi-terabyte production corpora, plan the surrounding operating layer too: durable ingestion queues, parser/OCR workers, external lexical indexes, tenant/ACL filters, index-lag monitoring, and backup/restore for each store.

Quick Start

Option A: Two lines (easiest)

import asyncio
from synaptic import SynapticGraph

async def main():
    # CSV file
    graph = await SynapticGraph.from_data("products.csv")
    try:
        result = await graph.search("my question")
        for activated in result.nodes[:5]:
            print(activated.node.title, activated.activation)
    finally:
        await graph.close()

asyncio.run(main())

You can pass preset="rag" to read SYNAPTIC_EMBED_URL and SYNAPTIC_RERANK_URL, or use GraphBuildOptions when you want one reusable configuration object across from_data(), from_chunks(), and from_database().

Option B: MCP server (Claude Desktop / Code)

synaptic-mcp --db my_graph.db
synaptic-mcp --db my_graph.db --embed-url http://localhost:11434/v1

Claude can now call MCP tools to explore your graph — search, ingest new files into the graph mid-conversation, and sync from a live database without dropping to a CLI.

A ready-to-paste claude_desktop_config.json snippet is in examples/mcp_claude_desktop.json.

Option BX: LangChain retriever (drop-in)

pip install "synaptic-memory[sqlite,korean,vector,langchain]"

import asyncio
from synaptic import SynapticGraph
from synaptic.integrations.langchain import SynapticRetriever

async def main():
    graph = await SynapticGraph.from_data("./docs/")
    try:
        retriever = SynapticRetriever(graph=graph, k=5)
        docs = await retriever.ainvoke("my question")
        for doc in docs:
            print(doc.page_content[:80], "   ", doc.metadata["score"])
    finally:
        await graph.close()

asyncio.run(main())

Runnable example: examples/langchain_retriever.py. Each hit becomes a LangChain Document with the node id, title, score, and any structured properties in metadata — works unmodified in RetrievalQA chains, agents, and RAG graphs.

Option C: Full control

import asyncio
from synaptic.backends.sqlite_graph import SqliteGraphBackend
from synaptic.extensions.domain_profile import DomainProfile
from synaptic.extensions.document_ingester import DocumentIngester, JsonlDocumentSource

async def main():
    profile = DomainProfile.load("my_profile.toml")
    backend = SqliteGraphBackend("graph.db")
    await backend.connect()

    source = JsonlDocumentSource("docs.jsonl", "chunks.jsonl")
    ingester = DocumentIngester(profile=profile, backend=backend)
    await ingester.ingest(source)

asyncio.run(main())

Option D: relation ontology (multi-hop)

When your documents cite each other by a canonical identifier — statute article numbers, standard clause codes, manual section ids — Synaptic turns those citations into REFERENCES graph edges so the agent can follow them. Still zero LLM at index time — it is rule-based extraction, auto-derived from your corpus's own identifier values.

from synaptic import SynapticGraph
from synaptic.extensions.domain_profile import DomainProfile

# Each document carries its identifier in an `article_no` property,
# scoped by `law`. That is all the configuration needed.
graph = await SynapticGraph.from_data(
    "./statutes.jsonl",
    profile=DomainProfile.with_references(
        key_property="article_no", scope_property="law"
    ),
)

The linker is self-gating: if the corpus has no clean identifier inventory it writes nothing (no-op), so this is safe to leave on. Measured impact — on a financial-statute multi-hop corpus, retrieval goes from 32% (standard dense RAG) / 31% (HippoRAG2) to 73% (synaptic) — 2.3× the nearest competitor: docs/REPORT-rag-vs-synaptic.md.

Requirement: each document must carry the identifier as a node property. Structured input (JSONL with properties) provides it directly; raw text files do not — relation linking needs identifiers.

Indexing cost comparison

By default, indexing is LLM-free. The graph is a search index, not a knowledge base. Cross-references that documents state explicitly (statute article citations, clause codes) are turned into REFERENCES edges with zero LLM (see Option D). If you opt into OpenIE, Synaptic can add bounded, revertible LLM-extracted semantic relations; that is not part of the default deterministic path.

Current API: graph.search() has one retrieval path: the hybrid EvidenceSearch pipeline (BM25 + HNSW + PPR + cross-encoder + MMR). The old engine= switch was removed, so examples should call graph.search("question") directly.

Agent Tools

Text search tools

Structured data tools

All four structured tools emit hints on 0-result returns (alternate operator, dropped WHERE, fuzzy column match) so the agent's next turn gets a concrete corrective action instead of a retry loop.

Ingest / CDC tools

Mid-conversation ingestion so Claude can teach itself new material without leaving the chat.

Navigation tools

All tools return { data, hints, session }. The SearchSession tracks seen nodes across turns so the agent never re-reads the same chunk.

Retrieval Pipeline

Query
  ↓  Kiwi morphological analysis (Korean) or regex (other)
  ↓  BM25 FTS + title 3x boost + substring fallback
  ↓  Vector search (usearch HNSW, optional)
  ↓  Vector PRF (pseudo relevance feedback, 2-pass)
  ↓  PPR graph discovery (personalized pagerank)
  ↓  GraphExpander (1-hop: category siblings, chunk-next, entity mentions)
  ↓  HybridReranker (lexical + semantic + graph + structural + memory + authority + temporal)
  ↓  MaxP document aggregation (coverage bonus)
  ↓  Cross-encoder reranker (bge-reranker-v2-m3 via TEI, optional)
  ↓  EvidenceAggregator (MMR diversity + per-doc cap + category coverage)
Result

Usage/time memory axis (opt-in, off by default). The reranker carries a fifth weighted signal — memory — that scores each node by how it has been used: importance (reinforced successes vs failures), recency (updated_at), and vitality. With memory=0.0 (the default) ranking is unchanged. Turn it on and retrieval evolves — reinforcing the results that answered a query lifts them on later searches, and decayed nodes fade, which a static index cannot do.

from synaptic.extensions.hybrid_reranker import RerankerWeights

# Enable the memory axis (rebalance the others so weights still sum to ~1).
graph.reranker_weights = RerankerWeights(
    lexical=0.35, semantic=0.20, graph=0.10, structural=0.10, memory=0.25,
)
await graph.reinforce([node_id], success=True)  # this result helped → lift it next time

Memory operating layer

Retrieval can be observed without making every search stateful:

from synaptic import FeedbackSignal, MemoryScope

scope = MemoryScope(workspace_id="docs", user_id="alice")
result = await graph.search("refund exception", record=True, scope=scope)

await graph.record_feedback(
    event_id=result.event_id,
    signal=FeedbackSignal.EXPLICIT_POSITIVE,
    success=True,
    scope=scope,
)

health = await graph.memory_health(scope=scope)
signals = await graph.scan_memory_signals(scope=scope)

Events, feedback, provenance, and health signals are stored as graph metadata. They are not appended to Node.content, and they are not automatically dumped into LLM prompts.

Benchmarks And Reports

The root README stays current with the install path and public API. Detailed numbers are versioned in reports so old measurements do not look like the current API contract.

Run a quick local smoke:

synaptic-quickstart --json

Run the lightweight Korean FTS benchmark:

pip install "synaptic-memory[korean]"
python examples/benchmark_allganize.py

Run the competitor harness when optional packages/API keys are available:

python examples/benchmark_vs_competitors/run_comparison.py --only synaptic

Reference reports:

Architecture

SynapticGraph.from_data("./data/")          ← Easy API
  ↓
Auto-detect → DomainProfile → Ingest → Index
  ↓
StorageBackend (Protocol)
  ├── MemoryBackend        (testing)
  ├── SqliteGraphBackend   (recommended, FTS5 + HNSW)
  ├── KuzuBackend          (embedded Cypher)
  ├── PostgreSQLBackend    (pgvector)
  └── CompositeBackend     (mix backends)
  ↓
Retrieval pipeline (BM25 + vector + PRF + PPR + reranker + MMR)
  ↓
Agent tools → MCP server → LLM agent

Backends

Optional Extras

Documentation

Dev

uv sync --extra dev --extra sqlite --extra mcp
uv run pytest tests/ -q
uv run ruff check --fix

License

Apache-2.0 — see LICENSE. Permits commercial use, modification, and redistribution as long as the copyright/attribution notice is preserved.