เซิร์ฟเวอร์ MCP Searxng Crawl4ai | PlugMCP
ข้ามไปยังเนื้อหา ไดเรกทอรี ผลิตภัณฑ์ แหล่งข้อมูล Pricing จากผู้เผยแพร่
เอกสารประกอบ SearXNG + Crawl4AI MCP Server
A self-hosted MCP (Model Context Protocol) server providing fast search and reliable web scraping using SearXNG + Crawl4AI stack.
🚀 Why This Solution?
This project evolved from limitations found in self-hosted Firecrawl:
Self-hosted MCP server with SearXNG + Crawl4AI for fast search and reliable web scraping in Claude Code. 3x faster than native tools with 100% scraping reliability.
เสียบเครื่องมือใดก็ได้เข้ากับเอเจนต์ใดก็ได้ ไดเรกทอรี MCP สำหรับคนที่ไม่เขียนเซิร์ฟเวอร์
ผลิตภัณฑ์
ไดเรกทอรี
นักพัฒนา
บริษัท
❌ Firecrawl's search API doesn't work in self-hosted mode
❌ Missing Fire-engine features in self-hosted version
❌ Authentication issues and poor documentation
✅ Truly self-hosted search via SearXNG (aggregates 70+ search engines)
✅ Superior scraping via Crawl4AI (50k+ GitHub stars)
✅ 3x faster than Claude Code native search tools
✅ 100% reliable vs failing native WebFetch
✅ Complete privacy - no external API dependencies
🏗️ Architecture ┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ │ │ │ │ │
│ SearXNG │ │ Crawl4AI │ │ Redis │
│ (Search) │ │ (Scraping) │ │ (Cache) │
│ │ │ │ │ │
│ Port 8081 │ │ Port 8001 │ │ Port 6380 │
└─────────────┘ └──────────────┘ └─────────────┘
│ │ │
└───────────────────┼───────────────────┘
│
┌──────────────┐
│ │
│ MCP Server │
│ (TypeScript) │
│ │
└──────────────┘
│
┌─────────────┐
│ │
│ Claude Code │
│ │
└─────────────┘
📦 Features
🔍 Fast Search : SearXNG aggregates 70+ search engines (Google, Bing, DuckDuckGo, etc.)
🕷️ Advanced Scraping : Crawl4AI with Playwright for JavaScript-heavy sites
⚡ High Performance : Sub-second search, reliable scraping
🐳 Docker Ready : Complete Docker Compose orchestration
🔄 Proxy Support : Built-in rotating IP proxy integration
📊 MCP Integration : 3 powerful tools for Claude Code
🛡️ Privacy First : All processing happens locally
🚀 Quick Start
1. Clone and Setup git clone https://github.com/yourusername/searxng-crawl4ai-mcp
cd searxng-crawl4ai-mcp
npm install
npm run build
2. Start Docker Services # Start all services (SearXNG, Crawl4AI, Redis)
docker compose up -d
# Verify services are running
curl http://localhost:8081/search?q=test&format=json # SearXNG
curl http://localhost:8001/health # Crawl4AI
3. Configure Claude Code MCP Simple Configuration (No Proxy):
{
"mcpServers": {
"searxng-crawl4ai": {
"command": "node",
"args": ["fixed-mcp-server.js"],
"cwd": "/absolute/path/to/your/project"
}
}
}
With Proxy Configuration:
{
"mcpServers": {
"searxng-crawl4ai": {
"command": "node",
"args": ["fixed-mcp-server.js"],
"cwd": "/absolute/path/to/your/project",
"env": {
"PROXY_URL": "http://username:[email protected] :10000"
}
}
}
}
4. Increase Token Limits (Recommended) Create .claude/settings.json:
{
"environmentVariables": {
"MAX_MCP_OUTPUT_TOKENS": "100000"
}
}
🛠️ Available MCP Tools
1. search_web - Lightning Fast Search {
"query": "latest AI developments 2025",
"maxResults": 10
}
Returns: 30+ search results in <1 second from multiple engines
2. crawl4ai_scrape - Advanced Web Scraping {
"url": "https://finance.yahoo.com/quote/BTC-USD/",
"formats": ["markdown"]
}
Returns: Full page content with metadata (title, word count, clean markdown)
3. search_and_scrape - Combined Power Workflow {
"query": "Bitcoin technical analysis September 2025",
"maxResults": 2
}
Returns: Search results + scraped content from top URLs (complete market intelligence)
📊 Performance Benchmarks
🎯 Trading & Finance Use Cases Perfect for traders and financial analysts:
Real-time Price Data : Extract current Bitcoin, stock, forex prices with exact timestamps
Technical Analysis : Get complete RSI, MACD, support/resistance data from TradingView
Market Sentiment : Scrape Fear & Greed Index, VIX, sentiment indicators
News Analysis : Get latest Fed decisions, earnings, economic data
API Discovery : Extract trading APIs from financial websites
Use search_and_scrape to find "Bitcoin RSI technical analysis September 2025"
Result : Complete professional trading analysis with specific price levels, technical indicators, and market predictions.
🔧 Configuration
Environment Variables
Docker Services
SearXNG : Port 8081 - Metasearch engine
Crawl4AI : Port 8001 - Web scraping service
Redis : Port 6380 - Caching layer
🛡️ Security & Privacy
✅ No external API calls - everything runs locally
✅ Proxy support - hide your IP address
✅ Credential masking - sensitive data automatically masked in logs
✅ Self-hosted - complete control over your data
🆚 vs Alternatives
🚀 Advanced Usage
Proxy Configuration
Multiple Search Engines SearXNG automatically queries:
Google, Bing, DuckDuckGo
Startpage, Qwant, Yandex
Wikipedia, GitHub, StackOverflow
Academic sources (ArXiv, Google Scholar)
Custom Scraping Options {
"url": "https://example.com",
"formats": ["markdown", "html", "links"],
"wait_for": 2000,
"timeout": 30000
}
🐛 Troubleshooting
Services Not Starting docker compose logs searxng
docker compose logs crawl4ai
Port Conflicts Edit docker-compose.yml to change ports:
SearXNG: 8081 → your-port
Crawl4AI: 8001 → your-port
Redis: 6380 → your-port
MCP Connection Issues
Ensure all Docker services are running
Check absolute path in MCP configuration
Verify npm run build completed successfully
📄 License MIT License - Feel free to use in your projects!
🤝 Contributing Contributions welcome! Please read our contributing guidelines and submit pull requests.
⭐ Star This Repo If this MCP server helps your workflow, please star the repository!
Built with ❤️ for the Claude Code community
คะแนนในไดเรกทอรี 12/100 อยู่ในรายการ 10 จาก 10
อยู่ใน MCP Registry ทางการ 0 จาก 18
ผู้ดูแลอ้างสิทธิ์แล้ว 0 จาก 8
ตอบกลับพร้อมรายการเครื่องมือ 0 จาก 12
ดาวบน GitHub 10 จาก 16
ยอดดาวน์โหลดรายสัปดาห์ 0 จาก 10
SparkForge Access a versatile collection of tools for media generation, web scraping, and cryptocurrency research. Convert between data formats, summarize long documents, and extract text from images or PDFs wit Scrapling 🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and trick Scrapling MCP Server Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported. Fetch Fetch a web page and return it as clean markdown for the agent, in chunks for long pages. The reference server from the Model Context Protocol project. Obscura The headless browser for AI agents and web scraping Douyin TikTok Download API 🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API,支持 AI agent 通过 MCP 调用与离线语音转文字,Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI age Web Scout Search the web and extract clean, readable text from webpages. Process multiple URLs at once to speed up research with reliable throttling and error handling. Quickly compile sources and summaries for Apify MCP Server Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡ Apify The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automa Skill Seekers Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection SeleniumBase 📊 Python APIs for Web Automation, Testing, and Scraping. CDP Mode launches a stealth browser that bypasses bot-detection/CAPTCHAs. Adds stealth to Playwright and has an MCP server for AI agents. Mobile Model Context Protocol Server for Mobile Automation and Scraping (iOS, Android, Emulators, Simulators and Real Devices) เซิร์ฟเวอร์ MCP หมวด Web scraping เซิร์ฟเวอร์ MCP ที่ดีที่สุดในหมวด Web scraping
ยังไม่มีรายการเครื่องมือ เซิร์ฟเวอร์แบบรีโมตจะได้รายการหลังการตรวจสอบสถานะครั้งแรก
ติดตั้ง Searxng Crawl4ai อย่างไร? คัดลอกคอนฟิก stdio สำหรับ Cursor, Claude Desktop, Claude Code หรือ VS Code ได้ในหน้านี้ คอนฟิกนี้ใช้การติดตั้งของผู้เผยแพร่ ไม่ใช่พร็อกซีของ PlugMCP
Searxng Crawl4ai ต้องใช้ API key ไหม? ระเบียนนี้ไม่ได้ระบุตัวแปรสภาพแวดล้อมที่จำเป็น
Searxng Crawl4ai ใช้ทรานสปอร์ตแบบใด? Searxng Crawl4ai ทำงานแบบโลคัลผ่าน stdio
Searxng Crawl4ai ถูกตรวจสอบล่าสุดเมื่อใด? ตรวจสอบล่าสุดเมื่อ 2026-10-27 Self-hosted MCP server with SearXNG + Crawl4AI for fast search and reliable web scraping in Claude Code. 3x faster than native tools with 100% scraping reliability.
เครื่องมือมีคำอธิบาย 0 จาก 6