🕷️ MCP Web Scrape
Clean, cached web content for agents—Markdown + citations, robots-aware, ETag/304 caching.
Power your agent
Yayıncıdan kurulum. PlugMCP uç noktası değildir.
Host a copy with our keys and logs.
Registry'de henüz paketlenmiş bir sürüm yok. Deponun README dosyasını izleyin.
Uzak URL mi, yoksa makinenizde bir komut mu? İki kurulum arasındaki fark
PlugMCP’de çalıştır
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
BaşlatYayıncıdan
Clean, cached web content for agents—Markdown + citations, robots-aware, ETag/304 caching.
🚀 mcp-web-scrape — Clean, cache-aware web content fetcher for AI agents. Fetch any URL → extract readable content → return Markdown/JSON with citations. ⚡ Fast caching, 🤝 robots.txt compliant, 📝 Markdown-ready output, �� works with ChatGPT/Claude Desktop.
Current Version: 1.0.7
# Extract content from any webpage
npx [email protected]
# Example: Extract from a news article
> extract_content https://news.ycombinator.com
✅ Extracted 1,247 words with 5 citations
📄 Clean Markdown ready for your AI agent
# Extract all forms from a webpage
> extract_forms https://example.com/contact
✅ Found 3 forms with 12 input fields
# Parse tables into structured data
> extract_tables https://example.com/data --format json
✅ Extracted 5 tables with 247 rows
# Find social media profiles
> extract_social_media https://company.com
✅ Found Twitter, LinkedIn, Facebook profiles
# Analyze sentiment of content
> sentiment_analysis https://blog.example.com/article
✅ Sentiment: Positive (0.85), Emotional tone: Optimistic
# Extract named entities
> extract_entities https://news.example.com/article
✅ Found 12 people, 8 organizations, 5 locations
# Check for security vulnerabilities
> scan_vulnerabilities https://mysite.com
✅ No XSS vulnerabilities found, 2 header improvements suggested
# Analyze competitor SEO
> analyze_competitors ["https://competitor1.com", "https://competitor2.com"]
✅ Competitor analysis complete: keyword gaps identified
# Monitor uptime and performance
> monitor_uptime https://mysite.com --interval 300
✅ Uptime: 99.9%, Average response: 245ms
# Generate comprehensive report
> generate_reports https://website.com --metrics ["seo", "performance", "security"]
✅ Generated 15-page analysis report
# Install globally
npm install -g [email protected]
# Try it instantly (latest version)
npx mcp-web-scrape@latest
# Try specific version
npx [email protected]
# Or start HTTP server
node dist/http.js
Add to your ~/Library/Application Support/ChatGPT/config.json:
{
"mcpServers": {
"web-scrape": {
"command": "npx",
"args": ["[email protected]"]
}
}
}
Add to your ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"web-scrape": {
"command": "npx",
"args": ["[email protected]"]
}
}
}
Deterministic Results → Same URL always returns identical content
Smart Citations → Every fact links back to its source
Robots Compliant → Respects robots.txt and rate limits
Lightning Fast → ETag/304 caching + persistent storage
Agent-Optimized → Clean Markdown instead of messy HTML
# Install specific version
npm install -g [email protected]
# Or use directly (latest)
npx mcp-web-scrape@latest
# Or use specific version
npx [email protected]
# Environment variables
export MCP_WEB_SCRAPE_CACHE_DIR="./cache"
export MCP_WEB_SCRAPE_USER_AGENT="MyBot/1.0"
export MCP_WEB_SCRAPE_RATE_LIMIT="1000"
STDIO (default)
mcp-web-scrape
HTTP/SSE
node dist/http.js --port 3000
Access cached content as MCP resources:
cache://news.ycombinator.com/path → Cached page content
cache://stats → Cache statistics
cache://robots/news.ycombinator.com → Robots.txt status
We love contributions! See CONTRIBUTING.md for guidelines.
Good First Issues:
MIT © Mahipal
Built with ❤️ for the Model Context Protocol ecosystem
🚀 mcp-web-scrape — Clean, cache-aware web content fetcher for AI agents. Fetch any URL → extract readable content → return Markdown/JSON with citations. ⚡ Fast caching, 🤝 robots.txt compliant, 📝 Ma
Genel bakış
🚀 mcp-web-scrape — Clean, cache-aware web content fetcher for AI agents. Fetch any URL → extract readable content → return Markdown/JSON with citations. ⚡ Fast caching, 🤝 robots.txt compliant, 📝 Markdown-ready output, �� works with ChatGPT/Claude Desktop.
Araçlar
Henüz araç listesi yok. Uzak sunucular ilk sağlık kontrolünden sonra bir liste alır.
SSS
Bu sayfadan Cursor, Claude Desktop, Claude Code veya VS Code için stdio yapılandırmasını kopyalayın. PlugMCP proxy'sini değil, yayıncının kurulumunu kullanır.
Bu kayıtta gerekli bir ortam değişkeni listelenmiyor.
MCP Web Scrape yerel olarak stdio üzerinden çalışır.
Son kontrol: 2026-10-27. 🚀 mcp-web-scrape — Clean, cache-aware web content fetcher for AI agents. Fetch any URL → extract readable content → return Markdown/JSON with citations. ⚡ Fast caching, 🤝 robots.txt compliant, 📝 Markdown-ready output, �� works with ChatGPT/Claude Desktop.