The spider that learns.
接入你的智能体
从发布者处安装。不是 PlugMCP 端点。
远程 URL 还是本机命令? 两种安装方式的区别
来自发布者
A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent identity layer (Chrome TLS fingerprints, matched personas, vendor-aware Cloudflare/Akamai/DataDome handling) works to slip past bot detection. Ships with an MCP server so any LLM can drive a full crawl by conversation.
A self-healing web scraper for hostile sites — and it's driveable by any LLM.
Every scraper starts working. The question is how long before it breaks. Anansi is built on a different assumption: the web is adversarial and unstable, and your scraper should handle that without your involvement.
When a site changes its layout, Anansi finds the data anyway and remembers the fix. When a page needs a browser to render, it switches to one silently. When bot detection gets in the way, it presents a coherent identity — matched TLS fingerprint, persona, and headers — that works to slip past detection instead of tripping it. And when you re-crawl, unchanged pages are skipped before a request is even made. The result is a crawler that survives redesigns, handles hostile sites, and gets better the longer it runs.
Ships with an MCP server so any LLM can drive a full crawl through conversation.
<lastmod> filtering skip pages before the request goes out. Capabilities →item_schema and every scraped item is validated and coerced before it hits your database. Getting started →See the full capability reference for everything else — proxy rotation, adaptive rate limiting, URL canonicalization, JS interaction, network capture, and more.
# Core install
pip install "git+https://github.com/mdowis/anansi"
# For browser-based fetching (Cloudflare bypass, JS rendering):
playwright install chromium
# For TLS-fingerprint mimicry (curl-cffi impersonation):
pip install "anansi-scraper[tls] @ git+https://github.com/mdowis/anansi"
# Optional: the Patchright stealth browser engine (opt-in via ANANSI_BROWSER_ENGINE=patchright):
pip install "anansi-scraper[patchright] @ git+https://github.com/mdowis/anansi"
patchright install chromium
Full install matrix, extras, and Windows notes are in Getting started.
from pydantic import BaseModel
from anansi import Crawler
from anansi.core import Item, Request, Response
from anansi.spider.spider import Spider
class ProductItem(BaseModel):
title: str
price: float # "49.99" strings are auto-coerced
sku: str | None = None
class ShopSpider(Spider):
name = "shop"
start_urls = ["https://shop.example.com/products"]
item_schema = ProductItem # validate every yielded item against this model
async def parse(self, response: Response):
for link in response.css("a.product-link"):
yield Request(response.urljoin(link["href"]), callback="parse_product")
async def parse_product(self, response: Response):
yield Item({"title": response.css("h1")[0].get_text(), "url": response.url})
# Self-healing, browser auto-upgrade, adaptive rate limiting, and incremental
# re-crawls are all on by default.
crawler = Crawler(ShopSpider, concurrency=10, max_pages=1000)
async for item in crawler.run():
print(item.data)
More examples — structured-data extraction, pausing and resuming, proxies, exporting — in Getting started.
Anansi ships a FastMCP server exposing 17 scraping tools over stdio or SSE, so an LLM agent can run a full crawl through conversation:
# Register with Claude Code
claude mcp add anansi -- anansi-mcp
Works with Claude Code, Claude Desktop, Cursor, Windsurf, ChatGPT, LangChain, and the OpenAI Agents SDK. Setup for each is in the MCP server guide.
Anansi is a powerful scraping tool. You are solely responsible for how you use it. Before
scraping any site, ensure you have the right to access and use the data and that you comply with
the site's Terms of Service, its robots.txt, applicable rate limits, and all relevant laws
(including computer-misuse statutes such as the CFAA and data-protection law such as GDPR/CCPA).
The anti-bot, TLS-fingerprint-impersonation, and Cloudflare-handling features are intended for
authorized testing, research, and scraping of content you have the right to access — not for
circumventing access controls without permission. See DISCLAIMER.md for the
full statement and Anti-bot & identity for operator
controls.
Licensed under the Apache License, Version 2.0 — see LICENSE and
NOTICE. Use of this software is additionally subject to the acceptable-use terms in
DISCLAIMER.md.
A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent identity layer (Chrome TLS fingerprints, matched personas, vendor-a
概览
A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent identity layer (Chrome TLS fingerprints, matched personas, vendor-aware Cloudflare/Akamai/DataDome handling) works to slip past bot detection. Ships with an MCP server so any LLM can drive a full crawl by conversation.
工具
暂无工具列表。远程服务器在首次健康检查后会获得工具列表。
常见问题
在本页复制适用于 Cursor、Claude Desktop、Claude Code 或 VS Code 的 stdio 配置。它使用发布者的安装方式,而不是 PlugMCP 代理。
此记录未列出必需的环境变量。
Anansi 通过 stdio 在本地运行。
上次检查时间:2026-10-27。 A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent identity layer (Chrome TLS fingerprints, matched personas, vendor-aware Cloudflare/Akamai/DataDome handling) works to slip past bot detection. Ships with an MCP server so any LLM can drive a full crawl by conversation.