If you find this project helpful, please give us a ⭐️ on GitHub for the latest update.
🤝 Contributions welcome! Feel free to open an issue or submit a pull request.
TeleMem is an agent memory management layer that can be used as a high-performance drop-in replacement for Mem0 with one line of code (), deeply optimized for complex scenarios involving , , , and .
TeleMem is a high-performance drop-in replacement for Mem0, featuring semantic deduplication, long-term dialogue memory, and multimodal video reasoning.
Through its unique context-aware enhancement mechanism, TeleMem provides conversational AI with core infrastructure offering higher accuracy, faster performance, and stronger character memory capabilities.
Building upon this foundation, TeleMem implements video understanding, multimodal reasoning, and visual question answering capabilities. Through a complete pipeline of video frame extraction, caption generation, and vector database construction, AI Agents can effortlessly store, retrieve, and reason over video content just like handling text memories.
The ultimate goal of the TeleMem project is to use an agent's hindsight to improve its foresight.
TeleMem, where memory lives on and intelligence grows strong.
Why TeleMem?
🎭 Character memory done right — the only open-source memory layer that automatically builds isolated, per-character memory profiles, built for role-play, companion AI, NPCs, and multi-persona assistants.
🎬 Memory for video, not just text — a full video → frames → captions → vector DB pipeline with ReAct-style multi-step video QA.
🏠 Fully local by default — runs end-to-end on your hardware (Qwen + FAISS); no cloud service, no paid tier, no data leaving your machine.
🔌 mem0-compatible API — add() / search() accept the same arguments and return the same {"results": [...]} shapes, so existing Mem0 code keeps working.
📢 Latest Updates
[2026-08-15] 🎉 TeleMem v1.10.0 adds first-class DeepSeek Harness support: an opt-in Cordis/MCP patch starts TeleMem with uvx, registers all 8 memory tools under mcp__telemem__*, and securely forwards provider configuration. See the MCP server docs.
[2026-08-06] 🎉 TeleMem v1.9.0 is on the latest MCP standard: migrated to the official MCP Python SDK v2 (spec 2026-07-28) — all 8 tools now declare titles, behavior annotations, and structured output, while staying compatible with older MCP clients. See the MCP server docs.
[2026-07-11] 🎉 TeleMem v1.8.0 — the "claims = contracts" release: character-memory extraction fix, infer=False/prompt/memory_type now fully honored, offline contract test suite, telemetry disabled by default, and a multi-NPC demo!
[2026-06-12] 🎉 TeleMem v1.7.1 is live on the official MCP registry — run the memory server with zero install: uvx telemem! Also new: evaluation principles and a LongMemEval harness with built-in baselines.
[2026-06-12] 🎉 TeleMem is now on PyPI: pip install telemem! v1.6.0 adds Ollama/DeepSeek/Kimi configs, LangChain & LlamaIndex examples, and a documentation site.
[2026-06-12] 🎉 TeleMem v1.5.0 has been released: true mem0 drop-in API, lightweight core install, and CI!
TeleMem enables conversational AI to maintain stable, natural, and continuous worldviews and character settings during long-term interactions through a deeply optimized pipeline of character-aware summarization → semantic clustering deduplication → efficient storage → precise retrieval.
flowchart LR
A["Dialogue<br/>messages"] --> B["Character-aware<br/>summarization<br/>(global + per-character)"]
B --> C["Embedding +<br/>similar-memory<br/>retrieval"]
C --> D["Write buffer<br/>(batch flush)"]
D --> E["LLM semantic<br/>clustering & fusion"]
E --> F[("FAISS index +<br/>JSON metadata")]
Q["Query"] --> S["Vector search<br/>+ rerank"]
F --> S
S --> R["results"]
Features
Automatic memory extraction: Extracts and structures key facts from dialogues.
Semantic clustering & deduplication: Uses LLMs to semantically merge similar memories, reducing conflicts and improving consistency.
Character-profiled memory management: Builds independent memory archives for each character in a dialogue, ensuring precise isolation and personalized management.
Efficient asynchronous writing: Employs a buffer + batch-flush mechanism for high-performance, stable persistence.
Precise semantic retrieval: Combines FAISS + JSON dual storage for fast recall and human-readable auditability.
Applicable Scenarios
Multi-character virtual agent systems
Long-memory AI assistants (e.g., customer service, companionship, creative co-pilots)
Complex narrative/world-building in virtual environments
Dialogue scenarios with strong contextual dependencies
Video content QA and reasoning
Multimodal agent memory management
Long video understanding and information retrieval
TeleMem vs Mem0: Core Advantages
TeleMem deeply refactors Mem0 to address characterization, long-term memory, and high performance. Key differences:
Capability Dimension
Mem0
TeleMem
Multi-character separation
❌ Not supported
✅ Automatically creates independent memory profiles per character
Average dialogue length: 600 turns per conversation
Scenarios: daily interactions, plot progression, evolving character relationships
Memory capability was assessed via QA benchmarks, e.g.:
{
"question": "What is Zhao Qi's nickname for Bai Yulan? A Xiaobai B Xiaoyu C Lanlan D Yuyu",
"answer": "A"
},
{
"question": "What is the relationship between Zhao Qi and Bai Yulan? A Classmates B Teacher and student C Enemies D Neighbors",
"answer": "B"
}
Using uv (recommended — creates .venv from the committed uv.lock for a reproducible environment):
uv sync --all-extras # install TeleMem (editable) + all extras, incl. MCP
uv run python examples/quickstart.py
Or with conda + pip:
# Create and activate virtual environment
conda create -n telemem python=3.10
conda activate telemem
# Install from source (editable), with the extras you need
pip install -e ".[all]"
Example
Set your OpenAI API key:
export OPENAI_API_KEY="your-openai-api-key"
# python examples/quickstart.py
import telemem as mem0
memory = mem0.Memory()
messages = [
{"role": "user", "content": "Jordan, did you take the subway to work again today?"},
{"role": "assistant", "content": "Yes, James. The subway is much faster than driving. I leave at 7 o'clock and it's just not crowded."},
{"role": "user", "content": "Jordan, I want to try taking the subway too. Can you tell me which station is closest?"},
{"role": "assistant", "content": "Of course, James. You take Line 2 to Civic Center Station, exit from Exit A, and walk 5 minutes to the company."}
]
memory.add(messages=messages, user_id="Jordan")
results = memory.search("What transportation did Jordan use to go to work today?", user_id="Jordan")
for hit in results["results"]: # same result shape as mem0
print(hit["memory"])
Memory() uses the default provider settings inherited from mem0ai. To use the repository's local Qwen + FAISS configuration, load config/config.yaml explicitly:
from telemem.utils import load_config
import telemem as mem0
config = load_config("config/config.yaml")
memory = mem0.Memory(config=config)
The runnable examples also honor the same configuration through TELEMEM_CONFIG:
TeleMem supports MiniMax as an LLM backend via its OpenAI-compatible API.
A ready-to-use example config is provided at config/config.minimax.yaml.
export MINIMAX_API_KEY="your-minimax-api-key"
export OPENAI_API_KEY="your-openai-api-key" # still needed for embeddings
from telemem.utils import load_config
import telemem as mem0
config = load_config("config/config.minimax.yaml")
memory = mem0.Memory(config=config)
Key points for MiniMax usage:
LLM: MiniMax-M3.1-Flash (default) or MiniMax-M3, both with a 1,000,000-token context window and text/image/video input. Existing configs keep their explicit model IDs, including legacy and custom models.
Regional endpoints: use https://api.minimax.io/v1 (global) or https://api.minimax.cn/v1 (China) as openai_base_url; existing custom endpoint values are preserved.
Thinking: Flash always uses adaptive thinking, with max reasoning effort when omitted. The preset sets is_reasoning_model: true so mem0 forwards an optional reasoning_effort (low, medium, high, xhigh, or max) and omits its small default token cap. Flash rejects disabled thinking, reasoning_effort="none", service_tier, and reasoning_split=false. See the OpenAI-compatible API contract for details.
Tool history: when calling the LLM adapter directly, use mem0's response_callback to capture the complete SDK assistant message and append it with tool results on the next call. Preserve both reasoning_content and tool_calls; the adapter's simplified return value is not a complete assistant message.
Temperature: ignored by Flash and M3, which use a fixed value of 1.
Embeddings: MiniMax does not provide a public embedding API; configure a separate embedder (e.g. text-embedding-3-small) in the embedder section
More LLM Providers
TeleMem works with any OpenAI-compatible endpoint. Ready-to-use config examples ship in config/:
A single statement, or a list of dialogue messages with role (user/assistant) and content
user_id
Optional[str]
❌ No
Character/user to attribute the memory to; TeleMem keeps an independent memory profile per user_id. Omit it to store shared conversation-event memories
agent_id / run_id
Optional[str]
❌ No
Additional mem0-compatible scopes (e.g. one run_id per session)
metadata
Optional[Dict[str, Any]]
❌ No
Arbitrary metadata stored with each memory
infer
bool
❌ No
Extract salient facts with the LLM (default: True); False stores message contents verbatim with no LLM call
memory_type
Optional[str]
❌ No
Pass "procedural_memory" to create procedural memories via mem0's pipeline; omit for conversational memories
prompt
Optional[str]
❌ No
Custom extraction prompt (replaces the optimized default as the system prompt)
batch
bool
❌ No
Route through the high-throughput batched pipeline (add_batch)
Message preprocessing: Merge consecutive messages from the same speaker; normalize turn structure.
Multi-perspective summarization:
Global event summary
Character 1’s perspective (actions, preferences, relationships)
Character 2’s perspective
Vectorization & similarity search: Generate embeddings and retrieve existing similar memories.
Batch processing: When buffer threshold is reached, invoke LLM to semantically merge similar memories.
Persistence: Dual-write to FAISS (for retrieval) and JSON (for metadata).
🎭 Multi-character demo: examples/multi_npc.py runs five tavern
NPCs through one scene — a single add_batch(scene, user_id=[...]) call gives each NPC a
private memory profile plus a shared "events" world-state, and each NPC then recalls the
scene from their own perspective.
Search Memory (search)
Performs semantic vector-based retrieval of relevant memories with context-aware recall.
🔍 Search is based on FAISS vector retrieval, supporting millisecond-level responses.
Multimodal Extensions
Beyond text memory, TeleMem further extends multimodal capabilities. Drawing inspiration from Deep Video Discovery's Agentic Search and Tool Use approach, we implemented two core methods in the TeleMemory class to support intelligent storage and semantic retrieval of video content.
Method
Description
add_mm()
Process video into retrievable memory (frame extraction → caption generation → vector database)
search_mm()
Query video content using natural language, supporting ReAct-style multi-step reasoning
The same root output directory used by add_mm; it must contain exactly one captions/*/captions.json and one vdb/*/*_vdb.json
max_iterations
int
❌ No
Maximum MMCoreAgent reasoning iterations (default 15)
🛠️ ReAct-Style Reasoning Tools
search_mm internally uses MMCoreAgent, employing a THINK → ACTION → OBSERVATION loop with three specialized tools:
Tool Name
Function
global_browse_tool
Get global overview of video events and themes
clip_search_tool
Search for specific content using semantic queries
frame_inspect_tool
Inspect frame details within a specific time range
Multimodal Example
Run the multimodal demo:
python examples/quickstart_mm.py
On the first run, frames, captions and VDB JSON will be generated under the chosen output_dir. The repository ships a small sample video; generating captions and the video database still requires configured VLM and embedding services unless you already have these artifacts locally.
Complete code example:
import telemem as mem0
from pathlib import Path
from telemem.mm_utils.core import extract_choice_from_msg
# Initialize
memory = mem0.Memory()
# Define paths
repo_root = Path(__file__).resolve().parents[1]
video_path = repo_root / "data" / "samples" / "video" / "3EQLFHRHpag.mp4"
video_name = video_path.stem
output_dir = video_path.parent
# Step 1: Add video to memory (auto-processing)
vdb_json_path = output_dir / "vdb" / video_name / f"{video_name}_vdb.json"
if not vdb_json_path.exists():
result = memory.add_mm(
video_path=str(video_path),
output_dir=str(output_dir),
)
print(f"Video processing complete: {result}")
else:
print(f"VDB already exists: {vdb_json_path}")
# Step 2: Query video content
question = """The problems people encounter in the video are caused by what?
(A) Catastrophic weather.
(B) Global warming.
(C) Financial crisis.
(D) Oil crisis.
"""
messages = memory.search_mm(
question=question,
output_dir=str(output_dir),
max_iterations=15,
)
# Extract final answer
answer = extract_choice_from_msg(messages)
print(f"Answer: ({answer})")
MCP Server
TeleMem ships a Model Context Protocol (MCP) server, so any MCP-compatible client — Claude Desktop, Claude Code, Cursor, custom agents — can use TeleMem as its long-term memory.
Built on the official MCP Python SDK v2, the server implements the current MCP
specification (2026-07-28) while remaining compatible with older clients; every tool
declares titles, behavior annotations (read-only/destructive hints), and structured output.
The server exposes eight tools: add_memory, search_memories, get_memories, get_memory, update_memory, delete_memory, delete_all_memories, and memory_history. Calls without an explicit scope default to TELEMEM_DEFAULT_USER_ID (telemem-mcp); destructive bulk deletion always requires an explicit scope.
TeleMem itself collects no telemetry. The underlying mem0ai library ships
anonymized PostHog usage telemetry, which TeleMem disables by default
(import telemem sets MEM0_TELEMETRY=False unless you have already set it).
To opt back in:
export MEM0_TELEMETRY=true
Development and Contribution
Issues and pull requests are welcome — see the Contributing Guide to get started.
Changes between releases are tracked in the Changelog.
CI runs the offline test suite (uv run pytest tests/ -q) on Python 3.10–3.12 for every PR.
TeleMem’s development has been deeply inspired by open-source communities and cutting-edge research. We extend our sincere gratitude to the following projects and teams:
If you find TeleMem useful to your research or development, please cite our arXiv paper:
Chunliang Chen, Ming Guan, Xiao Lin, Jiaxu Li, Luxi Lin, Qiyi Wang, Xiangyu Chen, Jixiang Luo, Changzhi Sun, Dell Zhang, Xuelong Li. TeleMem: Building Long-Term and Multimodal Memory for Agentic AI. arXiv:2601.06037, 2026. https://arxiv.org/abs/2601.06037
@article{chen2026telemem,
title = {TeleMem: Building Long-Term and Multimodal Memory for Agentic AI},
author = {Chen, Chunliang and Guan, Ming and Lin, Xiao and Li, Jiaxu and Lin, Luxi and Wang, Qiyi and Chen, Xiangyu and Luo, Jixiang and Sun, Changzhi and Zhang, Dell and Li, Xuelong},
journal = {arXiv preprint arXiv:2601.06037},
year = {2026},
url = {https://arxiv.org/abs/2601.06037}
}
Citation metadata is also available in CITATION.cff (GitHub's "Cite this repository" button).
If you find this project helpful, please give it a ⭐️ — starring also keeps updates in your GitHub feed.
Made with ❤️ by Bloo-Mind AI Ltd and the Ubiquitous AGI team at TeleAI.
mcp-name: io.github.TeleAI-UAGI/telemem
T
Telemem
TeleAI-UAGI·494 estrelas·
TeleMem is a high-performance drop-in replacement for Mem0, featuring semantic deduplication, long-term dialogue memory, and multimodal video reasoning.
Ainda não há lista de ferramentas. Servidores remotos recebem uma após a primeira verificação de saúde.
Perguntas frequentes
Perguntas
Como instalo Telemem?
Copie nesta página a configuração stdio para Cursor, Claude Desktop, Claude Code ou VS Code. Ela usa a instalação do publicador, não um proxy do PlugMCP.
Telemem precisa de uma chave de API?
O publicador lista estas variáveis de ambiente: OPENAI_API_KEY, TELEMEM_CONFIG, TELEMEM_DEFAULT_USER_ID.
Qual transporte Telemem usa?
Telemem roda localmente via stdio.
Quando Telemem foi verificado pela última vez?
Última verificação: 2026-10-27. TeleMem is a high-performance drop-in replacement for Mem0, featuring semantic deduplication, long-term dialogue memory, and multimodal video reasoning.