WSO2 Docs MCP Server
"This is an unofficial community project. Not affiliated with or endorsed by WSO2."
A production-ready Model Context Protocol (MCP) server that provides AI assistants (Claude Desktop, Claude Code, Cursor, VS Code) with semantic search over WSO2 documentation via Retrieval-Augmented Generation (RAG).
Under the hood, it uses a blazing-fast dual-ingestion engine:
- GitHub Native: Fetches raw Markdown directly from WSO2's public GitHub repositories via the Git Trees API (avoids web-scraping noise and rate limits)
- Web Crawl Fallback: For products without dedicated GitHub docs repos (like the WSO2 Library)
Architecture
Documentation Sources
Prerequisites
- Node.js ≥ 20
- Docker (for pgvector)
- Embeddings - no API key required by default:
- Ollama (recommended) - runs locally, model auto-downloaded on first run
- If Ollama is not running, the server automatically falls back to HuggingFace ONNX (in-process, also downloads automatically)
- Cloud providers are also supported: OpenAI, Google Gemini, Voyage AI
Quick Start
Choose the setup path that fits your use case:
- Install from npm - simplest, no cloning required
- Clone and build - for development or contributions
Install from npm
Install the package globally to get the wso2-docs-mcp-server, wso2-docs-crawl, and wso2-docs-migrate commands available system-wide:
npm install -g wso2-docs-mcp-server
Prefer no global install? You can use
npx wso2-docs-mcp-server,npx wso2-docs-crawl, andnpx wso2-docs-migratein every step below - just replace the bare command with itsnpxequivalent.
1. Start pgvector
Download the docker-compose.yml and start the database:
curl -O https://raw.githubusercontent.com/iamvirul/wso2-docs-mcp-server/main/docker-compose.yml
docker compose up -d
2. Start Ollama (optional but recommended)
Install Ollama and pull the default embedding model:
ollama pull nomic-embed-text
ollama serve
No Ollama? Skip this step. The server automatically falls back to HuggingFace ONNX - model downloads on first use with no extra setup.
3. Run database migration
DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
wso2-docs-migrate
Run migration again whenever you change
EMBEDDING_DIMENSIONS(i.e. switch embedding provider). The script detects and handles dimension changes automatically.
4. Index WSO2 documentation
# Index all products (first run downloads the embedding model automatically)
DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
wso2-docs-crawl
# Index a single product (faster, great for testing)
DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
wso2-docs-crawl --product ballerina --limit 20
# Force re-index even unchanged pages
DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
wso2-docs-crawl --force
Available product IDs: apim, mi, bi, choreo, is, ballerina, library
5. Configure your AI client
The MCP server is launched on demand by your AI client - no background process needed.
Claude Desktop - edit ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"wso2-docs": {
"command": "wso2-docs-mcp-server",
"env": {
"DATABASE_URL": "postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs",
"EMBEDDING_PROVIDER": "ollama"
}
}
}
}
Claude Code - run once in your terminal:
claude mcp add wso2-docs \
--transport stdio \
-e DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
-e EMBEDDING_PROVIDER="ollama" \
-- wso2-docs-mcp-server
# Verify
claude mcp list
Cursor - create .cursor/mcp.json in your project root:
{
"mcpServers": {
"wso2-docs": {
"command": "wso2-docs-mcp-server",
"env": {
"DATABASE_URL": "postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs",
"EMBEDDING_PROVIDER": "ollama"
}
}
}
}
VS Code - create .vscode/mcp.json:
{
"servers": {
"wso2-docs": {
"type": "stdio",
"command": "wso2-docs-mcp-server",
"env": {
"DATABASE_URL": "postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs",
"EMBEDDING_PROVIDER": "ollama"
}
}
}
}
Using
npxinstead of global install? Replace"command": "wso2-docs-mcp-server"with"command": "npx"and add"args": ["-y", "wso2-docs-mcp-server"].
Cloud embedding provider? Add the key to
env, e.g."EMBEDDING_PROVIDER": "openai", "OPENAI_API_KEY": "sk-...".
Clone and build
1. Clone and install
git clone https://github.com/iamvirul/wso2-docs-mcp-server.git
cd wso2-docs-mcp-server
npm install
2. Start Ollama (optional but recommended)
Install Ollama and start it:
ollama serve
No Ollama? Skip this step. The server detects Ollama is not running and automatically falls back to HuggingFace ONNX inference - the model downloads on first use with no extra setup.
3. Configure environment
cp .env.example .env
# Defaults work out of the box with Ollama.
# Only edit if using a cloud provider (OpenAI / Gemini / Voyage).
4. Start pgvector
docker compose up -d
# pgAdmin available at http://localhost:5050 ([email protected] / admin)
5. Run database migration
npm run db:migrate
Note: Run migration again whenever you change
EMBEDDING_DIMENSIONS(i.e. switch embedding provider). The script detects and handles dimension changes automatically.
6. Index documentation
# Index all products
# On first run the embedding model is downloaded automatically (Ollama or HuggingFace)
npm run crawl
# Index a single product (faster, great for testing)
npm run crawl -- --product ballerina --limit 20
# Force re-index even unchanged pages
npm run crawl -- --force
7. Build and start the MCP server
npm run build
npm start
For development (no build step):
npm run dev
8. Configure your AI client
Replace
/ABSOLUTE/PATH/TO/wso2-docs-mcp-serverwith your actual clone path.
Claude Desktop - edit ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"wso2-docs": {
"command": "node",
"args": ["/ABSOLUTE/PATH/TO/wso2-docs-mcp-server/dist/src/index.js"],
"env": {
"DATABASE_URL": "postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs",
"EMBEDDING_PROVIDER": "ollama"
}
}
}
}
Claude Code:
claude mcp add wso2-docs \
--transport stdio \
-e DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
-e EMBEDDING_PROVIDER="ollama" \
-- node "/ABSOLUTE/PATH/TO/wso2-docs-mcp-server/dist/src/index.js"
# Verify
claude mcp list
See config-examples/claude_code.sh for a convenience script.
Cursor - create .cursor/mcp.json - see config-examples/cursor_mcp.json.
VS Code - create .vscode/mcp.json - see config-examples/vscode_mcp.json.
MCP Tools
Example response
[
{
"title": "Deploying WSO2 API Manager",
"snippet": "WSO2 API Manager can be deployed in various topologies…",
"source_url": "https://apim.docs.wso2.com/en/latest/install-and-setup/...",
"product": "apim",
"section": "Deployment Patterns",
"score": 0.8712
}
]
Local Embeddings
The default EMBEDDING_PROVIDER=ollama runs entirely on your machine with no API key. The startup sequence is:
Is Ollama running?
├── Yes → Is model present?
│ ├── Yes → Ready (instant)
│ └── No → Pull via Ollama (streamed, runs once)
└── No → Download ONNX model from HuggingFace Hub (~250 MB, cached after first run)
and run inference in-process via @huggingface/transformers
Both paths use nomic-embed-text / Xenova/nomic-embed-text-v1 by default and produce identical 768-dim vectors, so you can switch between them without re-indexing.
Hardware acceleration (HuggingFace ONNX fallback)
When Ollama is not available, the server auto-detects the best compute backend:
Why q8 on Apple Silicon instead of CoreML/Metal?
CoreML compiles Metal shaders on first use (~20 min cold-start). For the typical chunk sizes produced by this server (6–20 chunks per page), the CPU↔GPU transfer overhead eliminates any inference gain. INT8 quantized inference on ARM NEON SIMD is consistently ~100× faster than fp32 CPU with zero cold-start cost.
Benchmark (Apple M-chip, Xenova/nomic-embed-text-v1):
fp32 CPU (before): ~1,000 ms/chunk (68 chunks ≈ 68 s of embedding)
q8 ARM NEON: ~9 ms/chunk (68 chunks ≈ 0.6 s of embedding) ← ~100× speedup
Note: For small crawls (≤ 10 pages) total wall-clock time is dominated by network I/O (HTTPS fetches to docs sites), so the end-to-end improvement is modest. The embedding speedup becomes significant at scale - crawling 500+ pages where embedding previously accounted for hours of runtime. For best crawl performance, run Ollama (
ollama serve) which parallelises inference natively and has no per-chunk overhead.
Environment Variables
Core
Ollama (default)
Cloud providers
Embedding dimension reference
Scheduled Re-indexing
# Run a one-off re-index (checks hashes, skips unchanged pages)
npm run reindex
# Or from the project directory using node-cron (runs daily at 2 AM)
DATABASE_URL=... node -e "
const { ReindexJob } = require('./dist/jobs/reindexDocs');
const job = new ReindexJob();
job.initialize().then(() => job.scheduleDaily());
"
Project Structure
src/
config/ env.ts · constants.ts
vectorstore/ pgvector.ts · schema.sql
ingestion/ crawler.ts · parser.ts · githubFetcher.ts · markdownParser.ts · chunker.ts · embedder.ts
server/ mcpServer.ts · toolRegistry.ts
jobs/ reindexDocs.ts
index.ts
scripts/
crawl.ts CLI ingestion pipeline
migrate.ts Dynamic schema migration
config-examples/ claude_desktop.json · claude_code.sh · cursor_mcp.json · vscode_mcp.json
docker-compose.yml
.env.example
Development
# Type-check
npx tsc --noEmit
# Run crawl with tsx (no build needed)
npm run crawl -- --product ballerina --limit 5
# Run server in dev mode
npm run dev
