A production-ready Model Context Protocol (MCP) server that provides seamless integration with the API. This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability.
Enable language models to perform advanced AI-powered web scraping with enterprise-grade reliabili…
This MCP server targets ScrapeGraph API v2 (https://v2-api.scrapegraphai.com/api), aligned 1:1 with
scrapegraph-py PR #84. Auth uses the
SGAI-APIKEY header. Environment variables mirror the Python SDK:
SGAI_API_URL — override the base URL (default https://v2-api.scrapegraphai.com/api)
SGAI_TIMEOUT — request timeout in seconds (default 120)
SGAI_API_KEY — API key (can also be passed via MCP scrapegraphApiKey or X-API-Key header)
Legacy aliases (still honored): SCRAPEGRAPH_API_BASE_URL for SGAI_API_URL, SGAI_TIMEOUT_S for SGAI_TIMEOUT.
Connect to our hosted MCP server - no local installation required!
[!WARNING]
The legacy MCP endpoint at https://mcp.scrapegraphai.com/mcp will be
deprecated soon. New integrations should use the replacement MCP endpoint:
https://sgai-mcp-main.onrender.com.
Claude Desktop Configuration (Remote)
Add this to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):
To run the MCP server locally for development or testing, follow these steps:
Prerequisites
Python 3.13 or higher
pip or uv package manager
ScrapeGraph API key
Installation
Clone the repository (if you haven't already):
git clone https://github.com/ScrapeGraphAI/scrapegraph-mcp
cd scrapegraph-mcp
Install the package:
# Using pip
pip install -e .
# Or using uv (faster)
uv pip install -e .
Set your API key:
# macOS/Linux
export SGAI_API_KEY=your-api-key-here
# Windows (PowerShell)
$env:SGAI_API_KEY="your-api-key-here"
# Windows (CMD)
set SGAI_API_KEY=your-api-key-here
Running the Server Locally
You can run the server directly:
# Using the installed command
scrapegraph-mcp
# Or using Python module
python -m scrapegraph_mcp.server
The server will start and communicate via stdio (standard input/output), which is the standard MCP transport method.
Testing with MCP Inspector
Test your local server using the MCP Inspector tool:
The ScrapeGraph MCP server can be integrated with Google ADK (Agent Development Kit) to create AI agents with web scraping capabilities.
Prerequisites
Python 3.13 or higher
Google ADK installed
ScrapeGraph API key
Installation
Install Google ADK (if not already installed):
pip install google-adk
Set your API key:
export SGAI_API_KEY=your-api-key-here
Basic Integration Example
Create an agent file (e.g., agent.py) with the following configuration:
import os
from google.adk.agents import LlmAgent
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset
from google.adk.tools.mcp_tool.mcp_session_manager import StdioConnectionParams
from mcp import StdioServerParameters
# Path to the scrapegraph-mcp server directory
SCRAPEGRAPH_MCP_PATH = "/path/to/scrapegraph-mcp"
# Path to the server.py file
SERVER_SCRIPT_PATH = os.path.join(
SCRAPEGRAPH_MCP_PATH,
"src",
"scrapegraph_mcp",
"server.py"
)
root_agent = LlmAgent(
model='gemini-2.0-flash',
name='scrapegraph_assistant_agent',
instruction='Help the user with web scraping and data extraction using ScrapeGraph AI. '
'You can convert webpages to markdown, extract structured data using AI, '
'perform web searches, crawl multiple pages, and automate complex scraping workflows.',
tools=[
MCPToolset(
connection_params=StdioConnectionParams(
server_params=StdioServerParameters(
command='python3',
args=[
SERVER_SCRIPT_PATH,
],
env={
'SGAI_API_KEY': os.getenv('SGAI_API_KEY'),
},
),
timeout=300.0,)
),
# Optional: Filter which tools from the MCP server are exposed
# tool_filter=['scrape', 'extract', 'search']
)
],
)
Configuration Options
Timeout Settings:
Default timeout is 5 seconds, which may be too short for web scraping operations
Recommended: Set `timeout=300.0
Adjust based on your use case (crawling operations may need even longer timeouts)
Tool Filtering:
By default, all registered MCP tools are exposed to the agent (see Available Tools)
Use tool_filter to limit which tools are available:
tool_filter=['scrape', 'extract', 'search']
API Key Configuration:
Set via environment variable: export SGAI_API_KEY=your-key
Or pass directly in env dict: 'SGAI_API_KEY': 'your-key-here'
Environment variable approach is recommended for security
Usage Example
Once configured, your agent can use natural language to interact with web scraping tools:
# The agent can now handle queries like:
# - "Convert https://example.com to markdown"
# - "Extract all product prices from this e-commerce page"
# - "Search for recent AI research papers and summarize them"
# - "Crawl this documentation site and extract all API endpoints"
# Clone the repository
git clone https://github.com/ScrapeGraphAI/scrapegraph-mcp
cd scrapegraph-mcp
# Install dependencies
pip install -e ".[dev]"
# Set your API key
export SGAI_API_KEY=your-api-key
# Run the server
scrapegraph-mcp
# or
python -m scrapegraph_mcp.server
Testing with MCP Inspector
Test your server locally using the MCP Inspector tool:
Noch keine Tool-Liste. Remote-Server erhalten eine nach dem ersten Health-Check.
FAQ
Fragen
Wie installiere ich ScrapeGraphAI Scrapegraph?
Kopieren Sie auf dieser Seite die http-Konfiguration für Cursor, Claude Desktop, Claude Code oder VS Code. Sie nutzt die Installation des Herausgebers, keinen PlugMCP-Proxy.
Benötigt ScrapeGraphAI Scrapegraph einen API-Schlüssel?
In diesem Eintrag ist keine erforderliche Umgebungsvariable aufgeführt.
Welchen Transport verwendet ScrapeGraphAI Scrapegraph?
ScrapeGraphAI Scrapegraph wird über HTTP erreicht.
Wann wurde ScrapeGraphAI Scrapegraph zuletzt geprüft?
Zuletzt geprüft: 2026-11-07. Enable language models to perform advanced AI-powered web scraping with enterprise-grade reliabili…