Rein

rein

Multi-source cross-validated memory for AI agents

English

rein is a self-adaptive memory system for AI coding agents. It stores, recalls, and manages memories across sessions with embedding-based semantic dedup, data-driven decay (Kaplan-Meier survival curves), and a fully closed self-learning loop that replaces fixed parameters with learned values.

Current release: v1.5.1 (2026-10-20) — maintenance patch from a whole-project review of v1.5.0; no schema or config change. Consolidation keeps the evidence inherited from earlier merges and, like rein upgrade and merge refinement, never replaces a memory edited while the model ran; refinement and group synthesis send the whole text or fall back instead of truncating; dedup and vector search never mix vectors from two embedding models; recall re-applies topic, time and tier filters after canonical collapse and fusion keeps tied candidates; the reranker learns per request; HNSW saves and reindex single-flight are crash-safe; the Stop hook ingests the newest turns of a long transcript; proxy extraction handles streamed Responses text, split UTF-8 and every WebSocket turn; doctor --fix respects running workers; rein init writes the shapes VS Code and OpenCode read; a plain-HTTP remote Gemini endpoint disables the embedder instead of sending the key. 2077 lib tests / 0 fail; clippy + fmt clean.

v1.4.1 (2026-10-10) — maintenance patch on the v1.4.0 runway automation line. Tantivy no longer leaves one segment per memory: the staged rebuild loads the corpus through a single writer with automatic merges disabled and compacts to one segment (a 10.6k-memory database rebuilds in 15 s instead of ~28 min; 10,651 segments → 1), and single inserts wait for their scheduled merges. Background warmup runs on a detached thread, so rein serve exits on SIGINT/SIGTERM during a rebuild instead of lingering until it finishes. The GUI's synthesis thumbs no longer show a vote as recorded when the server refused the write (auth = "public" with a public_url rejects every REST mutation) and show the refusal reason instead; the Connectors page explains the owner-only 401; the GUI's own doctor page no longer flags gui_runtime by probing itself; and #[op] validates literal REST paths at expansion time. 1900 lib tests / 0 fail; clippy + fmt clean; 2 codex review rounds (1 P2 closed, last round clean).

v1.4.0 (2026-09-04) — the runway automation line. The adaptive pipeline now records every stage in SQLite (adaptive_pipeline_last_run, surfaced by rein doctor, adaptive-status and rein gc); A12 publishes a Complete generation even when the store moved during a multi-hour calibration (activation stays fail-closed on the input epoch); the leave-one-out corpus build uses an inverted index with semantics identical to the pairwise reference (two hours down to eight seconds on a 9.7k-memory database, verified against the reference on real data); rein warmup backfills missing vec_memories rows and migrates the table atomically when the embedding model changes, with a vec_rows_provenance stamp that refuses mixed-model writes; and [hooks.claude] gives Claude Code prompt-time recall context that emits RecallComplete with a feedback id. config_version 4. 1887 lib tests / 0 fail; clippy + fmt clean; 26 adversarial review rounds (14 P1 + 43 P2 closed, last two rounds clean).

A12 recall-fusion weights can now calibrate without human feedback, fully fail-closed: structural leave-one-evidence-out families (canonical evidence / concept / episode), a SHA-256 family-disjoint split with a permanent activation holdout, a side-effect-free recall trace, paired-McNemar holdout gating, and a sealed parameter policy (schema 3) that steps recall_fusion:* adoption by at most 0.05 per refresh behind the judge structural trust gate. A SQLite schema-v4 input-epoch counter binds every calibration to the exact store state it was trained on; structural judge probes give zero-human-pair deployments an honest health gate; destructive dedup merges are floored at the static threshold with an exact false-merge safety bound. Released ≠ activated: everything ships shadow/fail-closed until live evidence passes the release gate. 1819 lib tests / 0 fail; clippy + fmt clean; 7 adversarial review rounds.

Recent releases (v0.33 → v0.35) — the eval-gate harness went from foundation to full: the dedup / admission / latency gates moved from NoData stubs to working scorers with committed baselines and 20-fixture corpora per gate (v0.33.0/.1, v0.35.0). Trust & Measurement Phase 3 landed its first slice — repair_advice + judge_drift_alert_total (v0.35.0); claude.ai remote-MCP polish added a sliding session cookie + metadata JSON (v0.35.0); and the bearer-auth migration progressed from a rein doctor WARN on the legacy loopback bool (v0.34.0) to its load-time removal (v0.35.0).

Trust & Measurement Phase 2 (v0.32.0) — the eval::gates module: GateScorecard / Gate trait / compare_scorecards (8-rule classification + paired McNemar non-inferiority), the recall gate over a hermetic 20-fixture corpus, the rein-eval gate {baseline,run,compare,status} CLI, rein_trust_measurement reading real scorecards, and rein doctor check_eval_gates.

Earlier hardening arc (v0.30.x → v0.31.x, 8 releases, 30 codex audit rounds) — built-in OAuth 2.0 provider for Claude Cowork / claude.ai / mobile remote MCP (DCR + PKCE S256 + RFC 9728/8414, [server].auth policy, SQLite-backed clients/grants/signing keys), JSON-RPC envelope on /mcp 4xx responses for claude.ai UI surfaceability, recall-launch warmup fragility fixes + 23-round codex audit, OAuth security A-H1 (kid strict match) / A-H2 (migration schema gate) / A-H3 (30s SHA-256 bearer cache + 60s debounced mark_client_used), recovery-path tail (TantivyFts::open_existing / symlink chain cycle detection / stale .rebuilding TTL recovery / atomic_write_string chown preservation), and build-path hygiene (CARGO_ENCODED_RUSTFLAGS with --remap-path-prefix=$HOME=user). Tagged in git (cargo install --git ... --tag v0.30.0..v0.31.3).

Earlier (v0.28.x distribution arc) — [mcp_servers.<name>] Codex 0.129 compat (v0.28.15/16), rmcp 1.6 host-guard bridge (v0.28.13), --locked install footgun docs (v0.28.14), rein_feedback MCP inputSchema hotfix (v0.28.12), second-pass audit hardening on v0.28.7 (v0.28.8, 17 codex review rounds), [ars.acceleration] + runtime LLM judge + Trust & Measurement default-on (v0.28.6).

For the full GitHub-ready manual, see docs/manual/README.md. Reference tables live under docs/reference/.

Features

Installation

Three install paths depending on your client:

Install on Claude Desktop (DXT — macOS Apple Silicon)

rein ships as a Claude Desktop Extension (.mcpb). One-click install, no Rust toolchain required.

1. Download rein-v1.5.1.mcpb from the v1.5.1 release.

2. Clear macOS quarantine (one-time, the build is unsigned):

   xattr -d com.apple.quarantine ~/Downloads/rein-v1.5.1.mcpb
   

3. Double-click the file. Claude Desktop opens its install dialog.

4. Fill in Gemini API Key (required); leave Memory database path and Supermemory API Key blank to use defaults.

5. Click Install. Claude Desktop spawns rein serve over stdio. ~40 rein_* tools appear in your next chat.

For step-by-step including upgrade, uninstall, and troubleshooting, see docs/manual/02-installation.md → Claude Desktop. For maintainers, the build pipeline is documented in docs/guides/dxt-build.md.

Other platforms (Intel Mac / Linux / Windows): the DXT bundle is macOS Apple Silicon only. Use the Claude Code plugin marketplace path or cargo install instead.

Install via Claude Code plugin marketplace

/plugin marketplace add lyr1cs/rein
/plugin install rein@rein

The plugin registers the rein MCP server entry in Claude Code. You still need the rein binary on your PATH:

# Latest master
cargo install --git https://github.com/lyr1cs/rein --locked rein

# Or pin to a specific release tag (recommended for reproducible installs)
cargo install --git https://github.com/lyr1cs/rein --tag v1.5.1 --locked rein

--locked is required. Without it, cargo install --git ignores the committed Cargo.lock and re-resolves every transitive dependency to the latest semver-compatible version on crates.io, which can pull in C/SIMD code that requires a newer host toolchain than your machine ships. See docs/manual/02-installation.md → Remote Install.

This installs the default non-GUI binary. For the built-in OAuth remote-connector flow, use the GUI-enabled source install in Recipe E so the owner approval pages are embedded.

Then set GEMINI_API_KEY in your shell environment or ~/.rein/config.toml. See docs/manual/02-installation.md for full configuration.

Install via Remote MCP (Cowork, claude.ai, mobile)

Claude Cowork (the agentic-work tab inside Claude Desktop), claude.ai web, and the Claude mobile apps route MCP traffic through Anthropic's cloud, not through your local stdio. They cannot use the DXT or plugin marketplace paths above. They need a public HTTPS endpoint that Anthropic's servers can reach.

rein already implements the Streamable HTTP transport (current MCP standard) at /mcp on its built-in HTTP server. Three prerequisites before exposing it:

1. Set [server].auth = "public" in ~/.rein/config.toml (loopback bind, non-discoverable public hostname will forward through) OR set REIN_HTTP_TOKEN to a strong secret and have your reverse proxy inject Authorization: Bearer … upstream. The other policies are "loopback_only" (strict local), "bearer_required" (token required), and "oauth" (built-in OAuth provider, recommended for multi-user remote MCP).

2. Add the public hostname to [server].allowed_hosts in ~/.rein/config.toml, e.g., allowed_hosts = ["rein.your-domain.com"]. Without this, rein returns 403 Host header is not allowed for any request whose Host: is not localhost / 127.0.0.1 / ::1, and your tunnel will appear broken even when up.

3. Start the listener:

   rein serve --sse        # listens on 127.0.0.1:8680/mcp
   

Without one of the auth postures in step 1, rein serve --sse exits immediately with REIN_HTTP_TOKEN must be set. Without step 2 the listener starts but rejects every tunnel request.

Session lifetime. rmcp (rein's MCP transport layer) expires a Streamable HTTP session after 300 s without a request. Since v1.4.2 that expiry is invisible to clients: rein keeps each session's initialize parameters in memory and in <db dir>/<db file name>.mcp-sessions-<bind>-<port>-<hash>.json, and a request that carries an expired or pre-restart Mcp-Session-Id is served after rmcp re-creates the session under the same id. Clients that never re-initialize on 404 Session not found (several SDKs don't, e.g. typescript-sdk #1708) keep working across idle gaps and rein serve restarts. A 404 still happens when the client itself sent DELETE /mcp, when more than 1024 sessions were seen since that one (oldest is dropped), or when the sidecar file was deleted or unreadable before a restart; in those cases the client must re-initialize as the spec requires.

Expose the listener publicly via one of (no Tailscale or domain required for the simpler ones):

• Cloudflare Quick Tunnel — cloudflared tunnel --url http://localhost:8680. No account, no domain. Random *.trycloudflare.com URL, ephemeral. Lowest friction; works on most networks (use --protocol http2 if your network blocks QUIC).
• ngrok — free account, no domain. URL ephemeral on free tier; reserved domains on paid. Tends to work on networks where Quick Tunnel can't.
• Tailscale Funnel — free account, no domain. Permanent *.ts.net URL. Most network-tolerant of the no-domain options.
• Cloudflare Tunnel + own domain — permanent URL under your domain, optional Cloudflare Access OIDC for real edge auth.
• Caddy / nginx + Let's Encrypt — self-hosted on your own VPS.

Then in Claude: Customize → Connectors → "+" → Add custom connector, paste your URL (e.g., https://rein.your-domain.com/mcp), optionally add OAuth credentials.

Step-by-step recipes for each tunnel option, the Claude UI configuration flow, and authentication tradeoffs are in docs/manual/02b-remote-mcp-deployment.md.

Note: custom remote connectors require a Pro / Max / Team / Enterprise Claude account. Free-plan users can use the local-stdio paths (DXT, plugin marketplace) but not Cowork.

From source

git clone https://github.com/lyr1cs/rein.git
cd rein

# Standard build (CLI + MCP server only)
cargo install --path crates/rein --locked

# Full build with Neural Wiki GUI (recommended)
cd crates/rein/gui && npm ci && npm run build && cd ../../..
cargo install --path crates/rein --locked --features gui

Or use the install script:

./scripts/install.sh

The install script builds the embedded GUI by default when npm is available. Set REIN_INSTALL_GUI=0 for a CLI-only install.

Prerequisites

• Rust toolchain (1.75+)
• Node.js + npm (for GUI builds or the default install script path)
• A Gemini API key (free tier: 1500 req/day)

GUI Service Management

# Start GUI server in background (listens on :8680)
rein gui on

# Stop GUI server
rein gui off

# Or run in foreground with MCP + GUI
rein serve --gui

# Open in browser
open http://localhost:8680

The GUI requires building with --features gui. Without it, the gui subcommand is available but serves no embedded assets.

Quick Start

# 1. Set your API key
export GEMINI_API_KEY="your-key-here"

# 2. Auto-configure all detected MCP clients
rein init

# 3. Start the MCP server (usually done by your client)
rein serve

CLI Reference

How Cleanup Works (Provenance-Preserving)

rein's cleanup pipeline is provenance-preserving: it never hard-deletes information. The process has three stages:

1. Consolidation — Groups topic variants (e.g., Docker Deployment / docker-deployment) and merges all memories within each group into a single high-quality canonical memory. Source memories become evidence records in the memory_evidence table, preserving their original content, timestamps, and keywords.

2. Dedup — Scans for content-level duplicates within each topic group using lexical similarity (Jaccard + containment) and optionally embedding cosine similarity. Matches above threshold are merged into the winner; the loser's unique lines are appended with provenance markers ([merged from <id> on <date>]) and the loser is recorded as evidence.

3. Adaptive refresh — After consolidation and dedup, the adaptive engine (M1-M6) runs: HDBSCAN re-clusters, survival curves rebuild, tier boundaries update, and alpha/threshold learning processes new events.

Every merge decision is logged in the dedup_decisions append-only ledger with winner/loser IDs, scores, relation type, confidence, and operator. This is rein's equivalent of Git's reflog — you can always trace how a canonical memory was formed.

# Preview what cleanup would do (safe)
rein cleanup --all --dry-run

# Run cleanup on a specific topic
rein cleanup "docker-deployment"

# Full store cleanup
rein cleanup --all

# Run cleanup through the worker entrypoint
rein worker cleanup --all

consolidate keeps the old rein consolidate <topic> -s "summary" flow, but also supports:

• --topics a,b,c to batch a named topic set
• --pattern 'rmcp*' to batch by glob
• --all to process every topic
• --merge-variants to group case/space/hyphen variants such as Docker Deployment / docker-deployment
• omitting --summary to let rein auto-generate a consolidated memory, using the configured LLM when available and a local fallback otherwise

Batch consolidation fans out LLM synthesis asynchronously and in parallel, then commits SQLite writes sequentially. Cleanup actions also emit adaptive feedback and refresh M1-M6 state after the batch completes.

Cleanup is now scoped-first:

• rein cleanup X, rein cleanup --topics ..., or rein cleanup --pattern ... only deduplicates the selected groups
• destructive full-store cleanup requires rein cleanup --all
• rein cleanup --dry-run previews the scope
• background-style cleanup is handled by rein worker cleanup ..., rein worker cleanup-queue, and the cleanup queue worker

Store-time gray-zone dedup now also uses a dedicated async queue:

• hot-path store creates the new memory without blocking on remote LLM verdicts
• a dedup-queue worker later resolves gray-zone pairs with structured LLM verdicts
• you can drain it manually with rein worker dedup-queue

Recall is now evidence-aware:

• canonical memories are ranked with support_count and source_diversity
• recall output includes lightweight evidence_preview
• rein evidence <canonical_id> or /api/memories/:id expands the full evidence list
• lower-confidence / lower-corroboration results can use evidence second-stage rerank

Adaptive learning now sees richer canonical signals:

• reranker learning uses support / diversity features
• alpha optimization uses KG / episode / support / diversity-aware candidate scoring
• Adaptive GUI surfaces cluster-level dedup / admission / promotion decisions

CJK dedup now uses a hybrid lexical strategy:

• jieba-rs adds Chinese word segmentation
• character bigrams remain enabled as a fallback for CJK and mixed technical text
• both token streams are combined before Jaccard / containment scoring

More detailed docs:

• docs/guides/canonical-read-model.md
• docs/guides/evidence-aware-recall.md
• docs/reference/adaptive-learning-signals.md

Audit / handoff commit chain:

• 8b9e747
• b358100
• b861a4f
• 1b0765a
• 45de919
• d92170a
• d7200b3

Operator inspection commands:

• rein canonicals shows canonical memories and their support/merge counters
• rein evidence <canonical_id> shows absorbed evidence snapshots
• rein dedup-log shows the recent dedup ledger

MCP Tools

When running as an MCP server (rein serve), Rein exposes 40 production MCP tools through the operation inventory. The authoritative list is maintained in docs/reference/mcp-tools.md, grouped as:

• Core memory: store, recall, update, forget, recent, topics, canonicals, evidence, stats, and health.
• Maintenance: GC, dedup, concept dedup, organize, consolidate, cleanup, resummerize, and archive summary refresh.
• Knowledge graph and temporal: memoir tools, concept state, concept summary refresh, timeline, and concept history.
• Adaptive, session, ARS, and judge: feedback, adaptive status, session ingest, synthesis judge, and concept-summary judge.

Knowledge Graph Relation Types

part_of, depends_on, related_to, contradicts, refines, alternative_to, caused_by, instance_of, superseded_by

LLM Extraction (v0.3)

rein uses LLM (Gemini 3.1 Flash Lite or local models via OMLX) for structured memory extraction. The hook system automatically builds a knowledge graph from coding sessions.

Architecture:

• hook_post — local pattern extraction (crash safety net) + buffer to session file
• hook_compact — record compact context for async extraction
• hook_stop — queue full session distillation: memories + concepts + links + episode summary
• hook_session_start / hook_prompt — optional Codex additionalContext injection from Rein's working surfaces
• hook_pre_tool_use / hook_permission_request — deny-only Codex guardrails for obviously destructive shell commands

Upgrade old memories:

rein upgrade --dry-run    # preview
rein upgrade              # convert all old memories to knowledge graph
rein upgrade --topic debug  # convert specific topic only

Configuration:

[extract]
provider = "google"    # or a vendor name ("openai", "deepseek", "openrouter", …), "omlx", or "none"

[extract.google]
model = "gemini-3.1-flash-lite"
max_input_chars = 0    # 0 = no truncation (1M token model)

[extract.openai]       # one table for every OpenAI-compatible vendor; "omlx" is an alias
endpoint = "http://localhost:11434/v1"  # Ollama, LM Studio, vLLM, etc. — omitted for a named vendor
model = "default"
max_input_chars = 16000

Self-Learning Quality System (v0.3.0)

rein automatically learns which memories are useful and which are noise, without human parameter tuning.

How it works:

1. LLM assigns quality_confidence (0-1) at extraction time — zero extra API cost
2. System tracks recall-then-access patterns to classify memories as "good" (used) or "bad" (recalled but unused)
3. Feature weights auto-adjust from data: utility, novelty, connectivity, recency
4. Adaptive admission threshold rises when recent quality is low, relaxes when high
5. GC prunes low-quality concepts whose source memories are recalled 5+ times but never accessed

No manual tuning needed — cold-starts with LLM judgment, data gradually takes over.

Based on: ICLR 2026 Admission Control, PropMem (Prosus), FActScore, MACLA Bayesian posteriors.

Canonical-First Recall

rein now treats canonical memories as the default read model:

• store-time dedup tries to merge gray-zone writes into an existing canonical when evidence already exists
• admission/novelty scoring uses the current canonical view, not raw topic fragments
• working-set and always-on surfaces are refreshed from persisted canonical memories
• recall returns canonical memories by default, with evidence_preview for absorbed observations
• detail endpoints and GUI panels expand the full supporting evidence on demand

For API compatibility, GET /api/memories/:id returns the legacy top-level memory fields and also includes:

• memory: the canonical memory payload
• evidence: supporting evidence snapshots

Temporal Knowledge Graph (v0.4.0)

rein now tracks when knowledge changes, not just what the current state is. Inspired by Zep/Graphiti 2025.

Capabilities:

• Concept revision history — every refine_concept auto-snapshots the old state before overwriting
• Episode nodes — each session creates an Episode linking to concepts and memories touched
• Temporal link validity — ConceptLink has valid_from/valid_until windows; expired links are skipped in BFS
• Contradiction detection — when a new definition differs significantly (sim < 0.3), old outgoing links are expired
• Temporal recall — rein_recall supports from/to date params for time-range filtering
• Timeline view — rein_timeline shows chronological events (episodes, concept changes, memory creation)
• Concept history — rein_concept_history shows how a concept's definition evolved over time

Example queries enabled:

• "What changed last week?" → rein_timeline --from 2026-03-19 --to 2026-03-26
• "When did concept X change?" → rein_concept_history --memoir rust --name ownership
• "What did I know about Y before March?" → rein_recall "Y" --to 2026-03-01

Autonomous Retrieval Routing (v0.4.0)

rein automatically classifies queries and routes them to the optimal search strategy — no configuration needed.

Classification is rule-based (zero LLM calls, sub-microsecond). MCP responses include [route: type] prefix for transparency. TA-Mem 2026 and MemR3 2025 are tracked as related memory-retrieval background, not as implemented retrieval controllers.

Adaptive Engine (v0.6.0+)

rein's core philosophy is to minimize fixed parameters through data-driven adaptation. Bootstrap defaults still exist for cold start and safety, but the adaptive engine moves fusion, decay, tiering, and threshold behavior toward observed feedback in the slow channel.

Pipeline: M4 → A1 → M3 → M5 → M2 → M6

Also:

• Embedding-based semantic dedup in GC slow channel (catches paraphrases Jaccard misses)
• Provenance-preserving merge — temporal anchors and unique details never lost
• Snapshot CAS — adaptive state saved with read-merge-write on version conflict

Recent releases

The v0.21 → v0.38.0 arc rebuilt rein around six axes: a unified operation registry, an adaptive read-side synthesis (ARS) stack with feedback-driven gates, secure remote MCP deployment for Claude clients, a reproducible eval-gate harness for Trust & Measurement, an algorithm + performance pass on the recall and dedup hot paths, and a schema-versioning foundation for safe forward migrations.

v0.32.0 keeps the v0.28.7 / v0.28.8 default surface unchanged: only [ars.acceleration] ships default-on (still fail-closed — learned parameters do not affect runtime until a healthy ars_parameter_policy promotes a canary with positive scoped adoption weights). The runtime LLM judge ([ars.llm_judge]) and its nightly_cron remain default-off per the v0.28 charter Non-Goal — operators must explicitly opt in (incurs LLM API spend). ARS content-generation features ([ars].concept_summary_enabled, recall_synthesis_enabled, cold_archive_enabled) and [resummerize].enabled remain operator-controlled. The new [server].auth policy added in v0.30.0 stays the explicit gate for HTTP/SSE exposure — loopback_only is the default, and remote-MCP deployments require an explicit choice between bearer_required, oauth, or public.

Architecture Diagrams

Memory Storage Flow

flowchart TD
    A[Input text / tool output] --> B[hook_post or rein_store]
    B --> C[LLM Extraction\nGemini Flash Lite / OMLX]
    C -->|LLM unavailable| D[Rule-based fallback\ntopic · summary · keywords · importance]
    C --> D2[postprocess\ndate detection · preference tagging]
    D --> D2
    D2 --> E{store_with_dedup\nBEGIN IMMEDIATE}
    E -->|sim ≥ cluster_threshold A1| F[Provenance-preserving merge\nloser → evidence record]
    E -->|sim in gray-zone| G[LLM dedup verdict\nasync dedup-queue]
    E -->|new memory| H[INSERT memories]
    H --> I[auto_link\nbidirectional related_ids]
    I --> J[evolve\nknowledge evolution]
    J --> K[HNSW + Tantivy index update\nfire-and-forget]
    K --> L[needs_vec_dedup flag\nfor GC slow-channel embedding dedup]
    F --> M[dedup_decisions ledger]
    G --> M

Recall Pipeline

flowchart TD
    Q[Query] --> CL[Query Classifier\n6 strategies · rule-based · 0 LLM calls]
    CL -->|strategy + alpha| EX[Query Expansion\nGemini / OMLX → 2-3 variants]
    EX --> P1[Channel 1: Tantivy BM25\nlocal · <1ms]
    EX --> P2[Channel 2: HNSW vector\nlocal ~5ms / Gemini API ~255ms]
    EX --> P3[Channel 3: KG FTS + BFS\nconcept land-and-expand]
    P1 --> FU[RRF / CC Fusion\nlearned alpha M2]
    P2 --> FU
    P3 --> FU
    FU --> TF[M5 Tier Filter\nCold excluded for non-Exploratory]
    TF --> SW[Strength Weighting\nper-cluster KM curve M3 → global prior → Ebbinghaus]
    SW --> RF[Multi-feature Rerank\n8 features · learned weights]
    RF -->|optional| LR[LLM Reranker\nGemini / OMLX · strong-signal bypass]
    RF --> CC[Canonical-first collapse\nevidence_preview attached]
    LR --> CC
    CC --> CV[Cross-validate\nSupermemory + auto-memory files]
    CV --> RES[Final results\nconfidence 95%/85%/62% by source count]

Compression (PreCompact Hook)

flowchart TD
    T[PreCompact trigger\nContext window approaching limit] --> HC[hook_compact\nrecord compact context]
    HC --> SB[Read session buffer\naccumulated tool outputs + turns]
    SB --> LE[LLM extraction\nmemories + concepts + links]
    LE --> WQ[Async memory queue\n~/.rein/memory_queue_<project>.jsonl]
    WQ --> BW[Background worker\nrein worker memory]
    BW --> SD[store_with_dedup\nper-memory dedup + merge]
    SD --> EP[Episode node created\nsession → concept_ids + memory_ids]
    EP --> TL[ConceptLink temporal validity updated\nvalid_from / valid_until]
    TL --> CL[Session buffer cleared\nready for next context window]

    style T fill:#f96,color:#000
    style EP fill:#6af,color:#000

Configuration

rein loads configuration with the following priority (highest wins):

1. Environment variables
2. TOML config file ($REIN_CONFIG or ~/.config/rein/config.toml)
3. Compiled-in defaults

Environment Variables

Providers and API keys (v1.5.0)

rein talks to two kinds of model APIs: Google Gemini (provider = "google") and any OpenAI-compatible endpoint (provider = "<vendor>"). A vendor name carries defaults for the endpoint, the key's environment variable, the auth header and request quirks; every default can be overridden per section. rein providers vendors prints the built-in list (OpenAI, Perplexity, DeepSeek, OpenRouter, Groq, Mistral, xAI, Together, Fireworks, Voyage, Jina, Cohere, Anthropic, Azure, NVIDIA, Cerebras, Hugging Face, DashScope, Zhipu, SiliconFlow, Moonshot, Volcengine, Ollama, LM Studio, vLLM, omlx) with the chat / embeddings support of each. Vendors without an embeddings API (DeepSeek, Groq, xAI, Anthropic, …) are rejected for [embedding] at load time.

Keys are read, in order, from the process environment, then from rein's own ~/.rein/secrets.env (mode 600, written by rein keys set), then from a plaintext api_key = "…" in config.toml (works, but rein providers flags it). The secrets file is what launchd services and the rein serve processes started by Claude Desktop or Codex see, so it is the recommended place.

rein providers vendors                                   # what you can pick
rein providers set embedding --provider perplexity --model pplx-embed-v1-4b --dimensions 2560
rein keys set PERPLEXITY_API_KEY --stdin < key.txt       # or interactive hidden prompt
rein providers status --probe                            # calls each provider once
rein warmup --reembed-all                                # embedding model changed → re-embed
rein providers set judge --provider openrouter --model anthropic/claude-sonnet-4.5
rein keys set OPENROUTER_API_KEY

The same wiring is shown by rein status, rein config, GET /api/providers and the GUI's AI Keys page, always with masked keys; the page can reveal a key behind an eye toggle for the owner session only.

Equivalent TOML:

[embedding]
provider = "perplexity"            # google | <vendor> | none
dimensions = 2560

[embedding.openai]                 # table name is "openai" (alias "omlx") for every OpenAI-compatible vendor
model = "pplx-embed-v1-4b"
# api_key_env = "PERPLEXITY_API_KEY"   # vendor default
# send_dimensions = true               # Matryoshka models
# query_input_type = "query"           # asymmetric models (Voyage, Jina, NVIDIA)
# document_input_type = "document"

[ars.llm_judge]
provider = "openrouter"            # judge-only provider; other surfaces untouched
[ars.llm_judge.openai]
model = "anthropic/claude-sonnet-4.5"

One embedding model is active per database; switching re-embeds everything (rein warmup --reembed-all), and the vec_rows_provenance stamp refuses mixed-model writes meanwhile.

config.toml

[database]
path = "auto"                          # "auto" = ~/.rein/memories.db

[embedding]
provider = "google"    # google | <vendor name> | none
dimensions = 3072

[embedding.google]
model = "gemini-embedding-001"

[embedding.openai]     # every OpenAI-compatible vendor; alias [embedding.omlx]
endpoint = "http://localhost:8000/v1"
model = "default"

[search]
rrf_k = 60.0
rrf_fts_weight = 0.3
rrf_vec_weight = 0.7
fusion_method = "rrf"      # or "cc" (Convex Combination, Bruch 2023)
cc_alpha = 0.5             # CC blend: alpha * sparse + (1-alpha) * dense

dedup_similarity = 0.70    # uses max(jaccard, containment) similarity
dedup_time_window_days = 7

[chunking]
max_tokens = 512
overlap_percent = 10
metadata_prefix = true

[sync]
supermemory_enabled = true
auto_memory_enabled = true
auto_memory_glob = "~/.claude/projects/*/memory/**/*.md"

[decay]
base_lambda = 0.06
ltm_beta = 0.8
stm_beta = 1.2
interval_hours = 24
prune_threshold = 0.05
stm_to_ltm_access_count = 5

[server]
compact = false
sse_enabled = false
sse_port = 8680
sse_bind = "127.0.0.1"

Database

The database is stored at ~/.rein/memories.db by default. rein auto-migrates from the old location if needed.

Override with the REIN_DB environment variable or the [database] path config key.

Hook Setup for Claude Code

Add the following to your Claude Code settings.json to enable automatic memory extraction:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "",
        "hooks": [
          { "type": "command", "command": "rein hook post", "timeout": 10 }
        ]
      }
    ],
    "PreCompact": [
      {
        "matcher": "",
        "hooks": [
          { "type": "command", "command": "rein hook compact", "timeout": 10 }
        ]
      }
    ],
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          { "type": "command", "command": "rein hook stop", "timeout": 30 }
        ]
      }
    ],
    "UserPromptSubmit": [
      {
        "matcher": "",
        "hooks": [
          { "type": "command", "command": "rein hook prompt", "timeout": 10 }
        ]
      }
    ]
  }
}

Hook behavior:

• PostToolUse -- local pattern extraction (crash safety net) + buffers for session-end batch processing
• PreCompact -- records compact context for the async memory pipeline
• Stop -- queues full knowledge extraction: memories + concepts + links + episode summary via async worker
• UserPromptSubmit -- optional prompt-time context. Off by default; enable it in ~/.rein/config.toml:

[hooks.claude]
inject_prompt_context = true
# "working_set" (hook buffers, no database recall) or "recall"
# (local fast recall over the memory database: FTS / KG / cached vectors,
# no LLM calls). "recall" emits a RecallComplete event and ends the injected
# context with a `rein_feedback: request_id=... memory_ids=...` line so the
# agent can report which memories it actually used.
prompt_context_source = "recall"
max_additional_context_chars = 1200

rein doctor reports which of these hooks are installed (claude_hooks).

Hook Setup for Codex CLI

Codex CLI hooks require hooks = true (Codex 0.129+) and either ~/.codex/hooks.json or inline [hooks] tables in ~/.codex/config.toml.

rein init now configures the Codex MCP entry and installs the Rein hooks:

[features]
hooks = true

{
  "hooks": {
    "SessionStart": [
      {
        "matcher": "*",
        "hooks": [
          { "type": "command", "command": "REIN_AGENT_LABEL=codex rein hook session-start", "timeout": 5 }
        ]
      }
    ],
    "PreToolUse": [
      {
        "matcher": "*",
        "hooks": [
          { "type": "command", "command": "REIN_AGENT_LABEL=codex rein hook pre", "timeout": 5 }
        ]
      }
    ],
    "PermissionRequest": [
      {
        "matcher": "*",
        "hooks": [
          { "type": "command", "command": "REIN_AGENT_LABEL=codex rein hook permission", "timeout": 5 }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "*",
        "hooks": [
          { "type": "command", "command": "REIN_AGENT_LABEL=codex rein hook post", "timeout": 10 }
        ]
      }
    ],
    "UserPromptSubmit": [
      {
        "hooks": [
          { "type": "command", "command": "REIN_AGENT_LABEL=codex rein hook prompt", "timeout": 5 }
        ]
      }
    ],
    "Stop": [
      {
        "hooks": [
          { "type": "command", "command": "REIN_AGENT_LABEL=codex rein hook stop", "timeout": 30 }
        ]
      }
    ]
  }
}

The Codex hook payload differs from Claude Code's payload. Rein understands the official Codex fields (hook_event_name, tool_input, tool_response, prompt, last_assistant_message, and transcript_path). PostToolUse and Stop feed the same async memory pipeline used by Claude Code hooks. PreToolUse and PermissionRequest are deny-only guardrails. SessionStart and UserPromptSubmit can emit official Codex additionalContext JSON when explicitly enabled:

[hooks.codex]
inject_prompt_context = true
inject_session_context = true
max_additional_context_chars = 4000

Remote Access via HTTP/SSE

Start rein with SSE transport for remote or multi-client access:

rein serve --sse

By default, the server binds to 127.0.0.1:8680.

To bind to a non-localhost address, you must set the REIN_HTTP_TOKEN environment variable for bearer token authentication:

export REIN_HTTP_TOKEN="your-secret-token"

Configure bind address and port in config.toml:

[server]
sse_enabled = true
sse_port = 8680
sse_bind = "0.0.0.0"    # requires REIN_HTTP_TOKEN

Transparent Proxy (v0.10.0)

rein can run as a transparent HTTP proxy that records LLM conversations without modifying requests. This works with any agent that supports base URL override.

Quick Start

# 1. Start the proxy (background)
rein serve --proxy &

# 2. Use with your agent
ANTHROPIC_BASE_URL=http://127.0.0.1:8690 claude       # Claude Code
codex -c 'model_providers.rein_proxy={ name = "Rein Proxy", base_url = "http://127.0.0.1:8690/v1", env_key = "OPENAI_API_KEY", wire_api = "responses", supports_websockets = false, env_http_headers = { "x-rein-token" = "REIN_PROXY_TOKEN" } }' -c 'model_provider="rein_proxy"'

Shell Aliases (recommended)

Add to ~/.zshrc or ~/.bashrc for convenience:

alias rein-proxy="rein serve --proxy &"
claudep() { REIN_PROXY_ACTIVE=1 ANTHROPIC_BASE_URL=http://127.0.0.1:8690 ANTHROPIC_CUSTOM_HEADERS="x-rein-token: ${REIN_PROXY_TOKEN:-}" claude "$@"; }
codexp() { REIN_PROXY_ACTIVE=1 codex -c 'model_providers.rein_proxy={ name = "Rein Proxy", base_url = "http://127.0.0.1:8690/v1", env_key = "OPENAI_API_KEY", wire_api = "responses", supports_websockets = false, env_http_headers = { "x-rein-token" = "REIN_PROXY_TOKEN" } }' -c 'model_provider="rein_proxy"' "$@"; }
codexsubp() { REIN_PROXY_ACTIVE=1 codex -c 'model_providers.rein_sub_proxy={ name = "Rein Subscription Proxy", base_url = "http://127.0.0.1:8690", requires_openai_auth = true, wire_api = "responses", supports_websockets = false }' -c 'model_provider="rein_sub_proxy"' -c 'chatgpt_base_url="http://127.0.0.1:8690/backend-api"' "$@"; }
codexsubpws() { REIN_PROXY_ACTIVE=1 codex -c 'model_providers.rein_sub_proxy_ws={ name = "Rein Subscription Proxy WS", base_url = "http://127.0.0.1:8690", requires_openai_auth = true, wire_api = "responses", supports_websockets = true }' -c 'model_provider="rein_sub_proxy_ws"' -c 'chatgpt_base_url="http://127.0.0.1:8690/backend-api"' "$@"; }

Then: rein-proxy to start, claudep, codexp, codexsubp, or codexsubpws to use. For ChatGPT-login Codex, codexsubp remains the recommended loopback entrypoint; smoke it with ./scripts/smoke_codexsubp.sh. For the websocket-enabled path, use codexsubpws or ./scripts/smoke_codexsubp_ws.sh. The codexsubp/codexsubpws provider overrides are generated from scripts/codexsubp_provider.toml.tmpl, which is the single source of truth for requires_openai_auth = true.

Codex CLI Config (alternative)

Configure Codex CLI permanently in ~/.codex/config.toml using a custom provider:

[model_providers.rein_proxy]
name = "Rein Proxy"
base_url = "http://127.0.0.1:8690/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"
supports_websockets = false
env_http_headers = { "x-rein-token" = "REIN_PROXY_TOKEN" }

model_provider = "rein_proxy"

This makes all Codex calls go through the rein proxy by default (requires proxy to be running).

Supported Agents

Note: Codex subscription/OAuth login proxying is not the same as the API-key Responses API proxy above. For API-key Codex, keep using codexp. For ChatGPT-login Codex, codexsubp is still the recommended loopback entrypoint today: it keeps requires_openai_auth = true, points chatgpt_base_url at the local rein proxy, and disables websocket transport so the first-party backend stays on the local record-only path. rein now also has an experimental websocket-enabled path (codexsubpws / smoke_codexsubp_ws.sh) that starts with websocket transport and relies on local 426 Upgrade Required fallback when upstream websocket is unavailable.

For ChatGPT-login Codex on loopback, codexsubp is the practical path today. It uses a custom provider with requires_openai_auth = true so Codex still uses ChatGPT login, but the provider itself points to the local rein proxy and disables websocket transport. chatgpt_base_url is also pointed at the local proxy so helper/discovery traffic (/wham/*, /connectors/*, /v1/agent/register, etc.) follows the same path. This keeps the subscription-login flow working