🕷️ Sitewalk: Technical SEO Site Audit
Repository: librecrawl-technical-seo-audit-mcp · the open-source Sitewalk engine and MCP server · Hosted Sitewalk → audit.adityaarsharma.com (ChatGPT plugin coming soon)
Power your agent
การติดตั้งจากผู้เผยแพร่ ไม่ใช่เอนด์พอยต์ของ PlugMCP
URL แบบรีโมตหรือคำสั่งบนเครื่องของคุณ? การติดตั้งสองแบบต่างกันอย่างไร
รันบน PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
เปิดจากผู้เผยแพร่
Repository: librecrawl-technical-seo-audit-mcp · the open-source Sitewalk engine and MCP server · Hosted Sitewalk → audit.adityaarsharma.com (ChatGPT plugin coming soon)
Self-hosted technical SEO audit MCP. 50+ checks, WAF detection, ephemeral. Built on LibreCrawl.
Run a complete on-site SEO audit on any website — straight from Claude, Cursor, Codex, or any Model Context Protocol (MCP) client. Unlimited pages · 50+ checks · PDF + CSVs · MIT-licensed · self-hosted · ephemeral by design.
Built on the open-source LibreCrawl engine, exposed through 38 MCP tools your AI assistant calls directly.
Model Context Protocol (MCP) is the open standard that lets AI assistants like Claude, Cursor, or Codex call external tools. Think of it as "USB for AI assistants" — you plug a tool in, the AI can use it. librecrawl-technical-seo-audit-mcp is one of those tools. Once installed, you just ask your AI assistant to audit a site, and it does. No GUI. No dashboard. No exports.
New to all this?
You: Audit https://acme.com — full site, no caps, give me the zip
Agent: → librecrawl_start_chunked_audit · polls until done · saves zip locally
You: Show me broken pages + broken external links + hreflang errors
Agent: → reads CSVs, prints filtered tables. Server already forgot the audit.
That's the product. Your AI assistant runs a full technical SEO audit for you. You get a branded PDF + 7 CSVs covering 50+ technical checks, ready to hand a client. The server wipes everything the moment you download.
No flags to remember, no caps to set. Anyone who runs:
librecrawl_start_chunked_audit(url="https://yoursite.com")
gets a full end-to-end audit by default:
Verified on real production sites: 1,942-page and 709-page WordPress sites, full coverage, origins healthy throughout.
Note: runs one audit at a time (single-tenant backend). For team use, queue audits sequentially — concurrent multi-audit routing is on the roadmap.
There are great desktop SEO crawlers (you know the ones). There are great cloud SEO suites. There was no AI-native crawler. librecrawl-technical-seo-audit-mcp fills that gap with five things no comparable open-source MCP server does:
38 MCP tools your agent calls directly. No GUI app to babysit, no SaaS dashboard to log into, no CSV exports to upload to ChatGPT. You just ask.
Most SEO MCP servers (SiteAudit MCP, AgentAEO, SE Ranking MCP) run synchronously and disconnect on sites over a few hundred pages. librecrawl-technical-seo-audit-mcp runs the crawl in a background worker thread, persists progress to SQLite WAL, and returns a session_id in under 2 seconds. Your agent polls a tiny status tool until done. 10,000-page enterprise sites work the same as 50-page blogs. Survives PM2 / MCP-client restarts mid-crawl.
Cloudflare, Akamai, DataDome, Imperva, and PerimeterX challenge pages are served as 200 OK but contain a JavaScript challenge instead of your content. Most crawlers report these as "page OK, all good". librecrawl-technical-seo-audit-mcp fingerprints the challenge in the response body and flags bot_block_challenge_detected. You see what's actually broken.
Additive-Increase / Multiplicative-Decrease — the same algorithm TCP congestion control uses. Error rate > 10% → halve chunk, double delay. p95 latency > 1.5× target → 1.5× delay. Clean signals → additive decrease. Polite by construction. No rate-limit blow-ups. No manual tuning. Respects robots.txt Crawl-Delay floor.
Once you download the zip, the server deletes the session row, every artifact file on disk, AND the upstream LibreCrawl crawl record. Per-audit server footprint after cleanup: 0 bytes, 0 rows. Auditing 50 client sites? Zero data persists where another operator could see it.
WeasyPrint, A4, page numbers, footer on every page. Open in any PDF viewer. No SaaS watermark. Hand it to a client as your work.
curl -fsSL https://raw.githubusercontent.com/adityaarsharma/librecrawl-technical-seo-audit-mcp/main/install.sh | bash
The installer asks 3 questions (target client, optional Google PageSpeed API key, optional GSC integration) and writes a ready-to-use MCP entry into your Claude / Cursor / Codex / Windsurf config. Done.
You don't need to be. If you can:
curl command above…you're done. The installer handles Python, Docker, the LibreCrawl backend, and your AI client config. First-audit-to-zip is under 10 minutes from cold start.
Three ways to run: the one-liner above (guided installer) · Docker (full stack, one command — best for self-hosting/VPS) · manual (Python venv + your own backend). New here? Start with the one-liner.
Brings up both the LibreCrawl engine and the MCP server, wired together and health-gated, with one command. Requires Docker + Docker Compose.
git clone https://github.com/adityaarsharma/librecrawl-technical-seo-audit-mcp.git
cd librecrawl-technical-seo-audit-mcp
docker compose up --build # MCP at http://127.0.0.1:5081/mcp
The mcp service waits for LibreCrawl to report healthy before it starts. First build takes a few minutes (LibreCrawl pulls Chromium for JS rendering). Audit zips land in ./reports. Full knobs in docs/CONFIGURATION.md.
Add to your client config (Claude Desktop / Code — HTTP transport):
{
"mcpServers": {
"librecrawl": {
"command": "npx",
"args": ["-y", "mcp-remote", "http://127.0.0.1:5081/mcp"]
}
}
}
Python 3.10+, plus a LibreCrawl backend (the Docker path above is the easiest way to get one).
git clone https://github.com/adityaarsharma/librecrawl-technical-seo-audit-mcp.git
cd librecrawl-technical-seo-audit-mcp
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
# Point the MCP at your running LibreCrawl backend, then start it:
LIBRECRAWL_URL=http://127.0.0.1:5080 python server.py
Add to your client config (Claude Desktop example):
{
"mcpServers": {
"librecrawl": {
"command": "npx",
"args": ["-y", "mcp-remote", "http://127.0.0.1:5081/mcp"]
}
}
}
Per-client setup (Claude Code, Cursor, Windsurf, Codex, Continue.dev) and every environment variable: docs/GETTING-STARTED.md · docs/CONFIGURATION.md.
missing_hsts · missing_csp · missing_x_frame_options · missing_x_content_type_options · missing_referrer_policy · x_robots_tag_vs_meta_mismatch · mixed_content
bot_block_challenge_detected — fingerprints Cloudflare · Akamai · DataDome · Imperva · PerimeterX
sitemap_url_noindex · sitemap_url_3xx · sitemap_url_disallowed_in_robots · sitemap_contains_canonicalized · sitemap_over_50k_urls · sitemap_over_50mb
missing_return_tag · missing_self_reference · missing_x_default · invalid_codes · to_noindex · to_broken · conflicts_lang_attr
canonical_chain_depth · canonical_to_relative · canonical_to_redirect · canonical_outside_head · bad_canonical
redirect_chains · meta_refresh_redirect · js_redirect · http_refresh_redirect
Article · Product · Recipe · FAQPage · BreadcrumbList · Event · JobPosting · VideoObject · HowTo · Organization · LocalBusiness · Person · Review · AggregateRating · Course · NewsArticle — validates schema.org spec AND Google Rich Results required fields. Handles @graph (Yoast / Rank Math / WPRM).
url_contains_space · url_multiple_slashes · url_non_ascii · url_underscores · url_repetitive_path · long_urls · uppercase_urls · url_params_heavy
non_descriptive_anchor_text · empty_anchor_text · anchor_image_no_alt · broken_bookmarks
internal_nofollow_outlinks · nofollow_only_inbound · follow_and_nofollow_mixed · orphan_pages
lazy_load_attr_missing · srcset_missing · image_dimensions_missing · next_gen_image_format · image_oversized_kb · missing_alt_pages · broken_img_pages
html_over_2mb · noscript_in_head · broken_or_invalid_html · dom_size_excessive · lorem_ipsum_detected
iframes_present · iframe_missing_title · missing_favicon · missing_html_lang · invalid_html_lang · missing_charset · missing_viewport
spider_trap_calendar · url_session_id_high_entropy · faceted_url_explosion
low_readability (Flesch) · long_sentences · passive_voice_pct · missing_terminal_punctuation · boilerplate_ratio · ai_tell_tokens_found (delve · unlock · seamlessly · leverage) · has_lorem_ipsum
outlinks_to_localhost (RFC1918 in production)
🔗 Every outbound URL HEAD/GET-validated into 17 status classes — ok · redirect · forbidden · not_found · timeout · dns_error · ssl_error · connection_refused · etc. Per-target: final URL after redirects, source pages, anchor text, response time, server header.
📈 GSC merge — pull Google Search Console data, call librecrawl_merge_gsc_data(crawl_id, gsc_data). URLs normalised before joining. Emits 4 extra CSVs: per-page-with-gsc · gsc-winners · gsc-losers (high impr + CTR <2%) · gsc-quick-wins (position 11–20 + impr ≥100).
This is a factual feature comparison. Prices were checked at publication and may have changed — see each vendor's site for current pricing. Brand names belong to their respective owners.
Reading guide: if you currently use a paid on-site crawler and your workflow is "crawl → export CSVs → analyse", librecrawl-technical-seo-audit-mcp covers that flow inside your AI assistant for £0 with no page caps. If your workflow depends on JavaScript-rendered SPAs, that's on the roadmap but not shipped yet — use the desktop tool for now.
Single zip, 9 files:
You: Audit https://example.com — full site, no caps
Agent: → librecrawl_start_chunked_audit(url=..., total_max_pages=10000)
returns session_id in <2s
→ polls librecrawl_audit_status every 25s
status: crawling, pages_done: 47, current_delay_ms: 250
status: crawling, pages_done: 312, last chunk p95: 480ms, err_rate: 0%
status: done, pages_done: 534, artifacts_ready: true
→ librecrawl_audit_zip(session_id)
returns base64 zip (9 files, 320 KB) + sha256
SAVES LOCALLY as example.com-1a2b3c4d.zip
→ librecrawl_audit_confirm_saved(session_id, sha256=<hash of saved file>)
hash matches, so the server wipes: session rows, files, zip, upstream crawl
You: Show me broken pages + broken external links
Agent: → unzips, reads per-page.csv (filters status_4xx OR status_5xx)
→ reads external-links.csv (filters not_found · forbidden · 5xx · timeout)
→ prints both tables
Local zip is the only copy. The server deletes nothing until the agent proves the save with a matching sha256, and unconfirmed zips are swept after an hour.
Not planned: keyword research, backlink analysis, SERP tracking. Those are different problems with different MCP servers (DataForSEO, etc.). This tool is laser-focused on technical on-site SEO crawling.
Open an issue to bump priorities or request a check.
One audit, start to finished zip — every step is an MCP tool your agent calls:
sequenceDiagram
actor You
participant Agent as AI assistant
participant MCP as librecrawl-mcp
participant LC as LibreCrawl engine
You->>Agent: "Audit https://acme.com — full site, give me the zip"
Agent->>MCP: librecrawl_start_chunked_audit(url)
MCP->>LC: start crawl on a background worker
MCP-->>Agent: session_id (returned in < 2s)
loop every ~25s until done
Agent->>MCP: librecrawl_audit_status(session_id)
MCP-->>Agent: crawling · pages_done · p95 latency · err_rate
end
MCP-->>Agent: status: done · artifacts_ready
Agent->>MCP: librecrawl_audit_zip(session_id)
MCP-->>Agent: base64 zip + sha256 (branded PDF + 7 CSVs)
Agent->>Agent: save locally, hash the saved bytes
Agent->>MCP: librecrawl_audit_confirm_saved(session_id, sha256)
MCP-->>Agent: hash matches, server wiped, per-step cleanup result
Agent-->>You: saved locally, cleanup passed
Two processes: a thin MCP wrapper your agent talks to, and the LibreCrawl engine that does the crawling.
flowchart TD
A["🖥️ MCP client<br/>Claude Code · Desktop · Cursor · Codex · Windsurf"]
A -->|streamable HTTP or stdio| B
subgraph MCP["🕷️ librecrawl-technical-seo-audit-mcp — server.py (FastMCP · 38 tools)"]
direction TB
B["runner.py<br/>background worker thread + AIMD controller"]
C["state.py<br/>SQLite WAL — session state"]
D["external_links · schema · content_audit<br/>extended_checks · sitemap_fill · pdf_report"]
E["libreclient.py<br/>typed wrapper to the upstream API"]
B --> C
B --> D
B --> E
end
E -->|HTTP :5080| F["🐍 LibreCrawl Flask backend<br/>single-tenant · crawls + extracts SEO data"]
Full component walk-through: docs/ARCHITECTURE.md.
All environment variables are optional — the defaults just work. Set them via your shell, docker compose, or your MCP client config.
📖 Full reference, per-client config, and transport details: docs/CONFIGURATION.md.
Chunked audit (95% of work):
librecrawl_start_chunked_audit · librecrawl_audit_status · librecrawl_audit_zip · librecrawl_audit_confirm_savedlibrecrawl_audit_pause · librecrawl_audit_resume · librecrawl_audit_cancel · librecrawl_audit_force_advancelibrecrawl_audit_artifacts · librecrawl_audit_pdf · librecrawl_report_contentSpecialist:
librecrawl_external_links_audit — re-run external-link validation on a specific crawllibrecrawl_schema_validate · librecrawl_schema_check · librecrawl_schema_auditlibrecrawl_merge_gsc_data · librecrawl_append_gsc_section — Google Search Console data mergelibrecrawl_pagespeed · librecrawl_pagespeed_audit · librecrawl_pagespeed_audit_all_crawl_pages — PageSpeed Insightslibrecrawl_site_check — instant site-level checklibrecrawl_internal_links_analysis · librecrawl_filter_issues · librecrawl_visualization_dataMaintenance:
librecrawl_wipe_everything — nuclear reset to zerolibrecrawl_brain_purge_audit — purge a single auditLegacy (kept for backwards compat, avoid for big sites):
librecrawl_audit · librecrawl_full_audit_strict · librecrawl_generate_report · librecrawl_export_results · librecrawl_get_status · librecrawl_get_settings · librecrawl_list_crawls · librecrawl_start_crawl · librecrawl_stop_crawl · librecrawl_pause_crawl · librecrawl_resume_crawl · librecrawl_resume_from_crawl_idDeeper guides live in docs/:
Code: MIT. Use it on client work, agency work, internal tools, anything. No attribution required (but appreciated). See LICENSE.
Trademarks. All third-party product names mentioned in this README (including any names referenced in the comparison table) are property of their respective owners. This project is not affiliated with, endorsed by, or sponsored by any third-party tool vendor. Comparisons are based on publicly available information at the time of writing and exist for the purpose of informing readers evaluating different categories of SEO tooling.
Discoverability keywords: seo audit mcp server · open-source seo crawler · self-hosted seo crawler · technical seo audit mcp · on-site seo audit tool · alternative to paid seo crawlers · free seo audit tool · seo crawler for claude · seo crawler for cursor · seo crawler for openai codex · seo crawler for windsurf · seo crawler for continue.dev · mcp server for seo · model context protocol seo · hreflang audit tool free · canonical chain checker · broken link checker unlimited · core web vitals audit cli · structured data validator command line · schema.org rich results validator · sitemap audit tool · sitemap orphan detection · WAF detection crawler · cloudflare challenge detector · security headers checker · CSP HSTS audit · google search console integration crawler · soft 404 detection · chunked crawler no timeout MCP · technical SEO audit api · python seo crawler · seo agency tool open source · ephemeral seo audit · agency-safe seo crawler · branded pdf seo report · seo audit cli tool · mit-licensed seo crawler · free site audit tool · enterprise seo crawler self-hosted · librecrawl mcp · librecrawl mcp server
Self-hosted technical SEO audit MCP. 50+ checks, WAF detection, ephemeral. Built on LibreCrawl.
เครื่องมือ
ยังไม่มีรายการเครื่องมือ เซิร์ฟเวอร์แบบรีโมตจะได้รายการหลังการตรวจสอบสถานะครั้งแรก
FAQ
คัดลอกคอนฟิก stdio สำหรับ Cursor, Claude Desktop, Claude Code หรือ VS Code ได้ในหน้านี้ คอนฟิกนี้ใช้การติดตั้งของผู้เผยแพร่ ไม่ใช่พร็อกซีของ PlugMCP
ระเบียนนี้ไม่ได้ระบุตัวแปรสภาพแวดล้อมที่จำเป็น
Librecrawl — Technical SEO Audit MCP Server ทำงานแบบโลคัลผ่าน stdio
ตรวจสอบล่าสุดเมื่อ 2026-11-09 Self-hosted technical SEO audit MCP. 50+ checks, WAF detection, ephemeral. Built on LibreCrawl.