smartmcp
PyPI package: smartmcp-router (CLI: smartmcp)
Intelligent MCP tool routing that reduces context bloat by serving only the tools your AI actually needs.
In my own setup with 8 MCP servers and 224 tools, every AI request was loading ~66,000 tokens of tool schemas before the model even started thinking. With smartmcp, that dropped to ~1,600 tokens. A 97% reduction on every request.
Token counts estimated at ~4 characters per token. Actual counts vary by model tokenizer.
Most MCP setups expose every tool from every server to the AI at once. With 5+ servers, that's 50–200+ tool schemas crammed into the context window before the AI even starts thinking. smartmcp fixes this.
smartmcp is a proxy MCP server that sits between your AI client and your upstream MCP servers. It indexes all available tools using semantic embeddings and exposes a fixed two-tool surface: search_tools to discover the right upstream tool and call_discovered_tool to invoke it. The tool list never changes mid-session, so smartmcp works even with clients that ignore notifications/tools/list_changed.
How it works
AI Client (Claude Desktop / Cursor / your agent)
↕ stdio
smartmcp (proxy server)
↕ stdio (one connection per server)
[github] [filesystem] [google-workspace] [git] [memory] [puppeteer] ...
smartmcp uses a two-phase flow: discover, then call.
Phase 1: Discovery
- On startup, smartmcp connects to all your configured MCP servers, collects every tool schema, and builds a FAISS vector index using sentence-transformer embeddings.
- The AI always sees exactly two tools:
search_toolsandcall_discovered_tool. It callssearch_toolswith a natural language query, for examplesearch_tools({ "query": "create a GitHub issue" }). - smartmcp runs semantic search across all indexed tools and finds the top-k matches.
search_toolsreturns structured JSON. Each match includes the exacttargetidentifier, the upstreamserverandname, thedescription, the relevancescore, and the full upstreaminput_schema.
Phase 2: Calling
- The AI reads the matching tool's
input_schemafrom the search response and constructs validarguments. - The AI calls
call_discovered_tool({ "target": "<exact target from search>", "arguments": { ... } }). smartmcp resolves the target (for examplegithub__create_issue), routes the call to the correct upstream server, and returns the result. Arguments are forwarded to the upstream tool unchanged.
The intelligence is in the discovery step. The AI still does its own parameter construction based on the schema returned by search_tools. smartmcp just narrows down which tools it sees, and the static two-tool surface means the flow works even with clients that snapshot tools/list once and never refresh.
Installation
pip install smartmcp-router
Requires Python 3.10+.
Recommended on macOS, especially Apple Silicon: install with pipx so SmartMCP gets an isolated environment that cannot inherit a stale torch or torchvision from other projects. This avoids the most common class of install issues (see Troubleshooting).
pipx install smartmcp-router
Quick start
1. Create a config file
Create a smartmcp.json with your upstream MCP servers (same format as Claude Desktop / Cursor config):
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/user/documents"]
},
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "ghp_your_token_here"
}
},
"slack": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-slack"],
"env": {
"SLACK_BOT_TOKEN": "xoxb-your-token-here"
}
}
},
"top_k": 5,
"embedding_model": "all-MiniLM-L6-v2"
}
2. Add smartmcp to your AI client
Add smartmcp as your single MCP server entry. It now manages all your upstream servers defined in smartmcp.json.
Claude Desktop
Add to your claude_desktop_config.json:
{
"mcpServers": {
"smartmcp": {
"command": "smartmcp",
"args": ["--config", "/path/to/smartmcp.json"]
}
}
}
Cursor
Add to your .cursor/mcp.json:
{
"mcpServers": {
"smartmcp": {
"command": "smartmcp",
"args": ["--config", "/path/to/smartmcp.json"]
}
}
}
Custom agents
Point your MCP client at smartmcp the same way you would any stdio MCP server:
smartmcp --config /path/to/smartmcp.json
3. Use it
Your AI now sees two tools: search_tools and call_discovered_tool. When it needs to do something, it searches and then invokes:
AI calls:
search_tools({ "query": "read files from disk" })smartmcp returns: JSON with up to top-k matches. Each match includes a
target(such asfilesystem__read_file), the upstream tool'sdescription, and the fullinput_schema.AI sees: The complete parameter definitions inside each match. It picks the right one, builds
argumentsagainst that match'sinput_schema, and callscall_discovered_tool({ "target": "filesystem__read_file", "arguments": { ... } }).smartmcp proxies the call to the filesystem server unchanged and returns the result.
Configuration reference
Why smartmcp?
- Less context waste: Instead of 100 tool schemas in every request, the AI sees a fixed two-tool surface and only inspects the few schemas returned by
search_tools. - Better tool selection: Semantic search finds the right tools even when the AI doesn't know the exact name.
- Full schema in search results: Each match returned by
search_toolsincludes the upstreaminput_schema, so the AI can construct calls correctly without any furthertools/listrefresh. - Client-agnostic: The tool list never changes mid-session, so smartmcp works with clients that ignore
notifications/tools/list_changed(such as Qwen Code and similar agents). - Works with any MCP server: If it speaks MCP over stdio, smartmcp can proxy it.
- Drop-in replacement: Replace your list of MCP servers with one smartmcp entry. No code changes needed.
- Graceful degradation: If some upstream servers fail to connect, smartmcp continues with whatever is available.
Troubleshooting
torchvision::nms does not exist or Could not import module 'BertModel'
If smartmcp crashes at startup with one of these errors:
RuntimeError: operator torchvision::nms does not exist
ModuleNotFoundError: Could not import module 'BertModel'. Are this object's requirements defined correctly?
This is a torch / torchvision ABI mismatch in your Python environment, not a smartmcp bug. SmartMCP itself does not depend on torchvision. The mismatch usually appears after pip install --upgrade walks transitive dependencies forward (newer sentence-transformers -> newer transformers -> newer torch) while leaving an older torchvision in place. Newer transformers eagerly imports the vision pipeline during BertModel autoload, so the mismatched torchvision extension fails to register the nms operator and the whole import chain breaks. Apple Silicon users see this more often because mixing arm64 and x86_64 wheels makes the mismatch more likely.
Try these fixes in order:
-
Remove
torchvision. SmartMCP does not need it:pip uninstall -y torchvision -
Reset the torch stack so torch and torchvision share an ABI:
pip uninstall -y torch torchvision torchaudio pip install --upgrade torch -
Recreate a clean venv (most reliable on Apple Silicon):
python3 -m venv ~/.venvs/smartmcp source ~/.venvs/smartmcp/bin/activate pip install --upgrade pip pip install smartmcp-router
Installing with pipx (see Installation) avoids this class of issue altogether by giving SmartMCP its own isolated environment.
Contributing
smartmcp is early-stage and actively improving. Contributions are welcome, especially around search accuracy, embedding strategies, and support for new transports.
If you have ideas, find bugs, or want to add features, open an issue or submit a PR on GitHub.
