SpiderHead is a working environment for an SEO specialist and their AI agent. Crawl a site,
keep every scan, find and prioritise issues, hand tasks to developers, recheck the fixes and report —
all on your own machine, in a native desktop app, in the terminal or through Claude over MCP.
One core, three faces. The desktop app, the seohead CLI and the MCP server share the same
Python core, projects and retained SQLite scans.
SpiderHead — SEO crawler & audit toolkit: local-first CLI, MCP server for AI agents and a desktop app. Retained scans, evidence-backed issues, Screaming Frog import.
Estrelas
4
Ferramentas
—
Disponibilidade
—
Projects, not one-off crawls. Scans, tasks, sources (Search Console, Metrika, Webmaster, GA4)
and history live together, so the next audit starts where the last one ended.
Local-first. No hosted account and no web dashboard; paid providers run only with your key and
your confirmation.
Three ways to use it
Interface
For
Start with
Desktop — SpiderHead Desktop for macOS, Windows and Linux
Crawling, issues, URL inspector, link graph, toolbox and project work in a native window
spiderhead is the primary command; seohead is a fully equivalent alias, and sf-analyzer
ships alongside. The seohead-seotools distribution is not published to PyPI yet, so that exact
command starts working once the upload lands. Today the same one-command install already works
from the GitHub release, which pipx puts into the same kind of isolated environment:
Both routes install the base dependencies only. Optional extras (mcp, reports, render, …)
can be added with pipx inject seohead-seotools <extra-packages>; the extras table in
SETUP.md names what each one unlocks.
Homebrew (planned)
The maintained formula source lives in
packaging/homebrew/spiderhead.rb — a pypi-style formula
pinned to the v3.2.0 release sdist with the full dependency closure; it passes brew style and
brew audit. The tap repository itself (PavloSEO/homebrew-tap) does not exist yet, so until
it is published the command below is planned, not working:
brew install PavloSEO/tap/spiderhead # planned — no tap published yet
SpiderHead Desktop
Ready-made desktop installers (macOS .dmg, Windows .exe setup, Linux .AppImage/.deb) are
attached to every release tag: look for
SpiderHead-<version>-<platform> files and their desktop-SHA256SUMS.txt. Until release
certificates are configured the builds are unsigned, so macOS Gatekeeper and Windows SmartScreen
warn on first launch.
The app lives in desktop/ and installs together with the core (Python 3.10 or newer):
With uv, uv sync --all-extras creates the same environment from
the committed uv.lock, and uv run seohead --help runs it. On Windows PowerShell, activate the
virtual environment with .venv\Scripts\Activate.ps1.
The repository is named spiderhead; the Python distribution is seohead-seotools.
spiderhead is the primary CLI name and seohead is a fully equivalent alias; the import
package remains seohead.
all installs every optional dependency; smaller extras (mcp, render, reports, pdf, cluster,
gsc, sitemap, tui, remote) are listed in SETUP.md.
Provider credentials and browser binaries are separate and never required for the core crawl.
Setup from zero covers versions, environment variables, Docker
(CONTAINERS.md) and Linux servers without a display (LINUX_VPS.md).
Quick start
1. Crawl a site, audit it, build a report
# Crawl locally. The URL cap is an explicit budget, not a claim about site size.
seohead crawl-site \
--url https://example.com \
--max-urls 500 \
--scan-out ./scans/audit.sqlite
# Format the retained audit as a working spreadsheet or a client document.
seohead report-build --audit ./scans/audit.sqlite --format xlsx --out audit.xlsx
seohead report-build --audit ./scans/audit.sqlite --format docx --out audit.docx
crawl-site is the primary collector: free, local, polite by default (about two requests per
second per host), resumable (RECOVERY.md) and configurable through
--config/--set (see seohead crawl-site --config-help and
the configuration inventory). It sends only bounded, read-only
requests to the site you name. report-build formats existing evidence as XLSX, DOCX, CSV,
Markdown, JSON or a bounded PDF overview (PDF_OVERVIEW.md); it never
invents findings.
2. Analyse existing Screaming Frog exports
# Offline: no SF installation, licence or request to the site is needed.
seohead sf run --exports-dir docs/examples/exports --out ./report --tasks
You get audit.json, audit.md and a prioritized tasks.md/tasks.json backlog with
reproduction steps, DOM positions and fixes. A separately installed, licensed SF CLI can also be
driven directly with sf run --crawl. Other crawlers' CSV exports are not drop-in compatible; they
are read through a versioned manifest (THIRD_PARTY_CRAWL_IMPORT.md).
3. Run it as a project with an agent
# These calls only create or inspect local project state; they do not crawl.
seohead project new --directory ./shop --target https://example.com/
seohead project checklist-init --directory ./shop
seohead project progress --directory ./shop
Register the MCP server (below), then ask the agent for the outcome you need, for example:
Audit this site for the agreed scope. Give the developers an Excel workbook of tasks, the
underlying exports and evidence links, proposed fixes, and acceptance/recheck criteria.
SpiderHead Desktop is a native PyQt5 application for macOS, Windows and Linux. It is a presentation
layer over the same core: it opens the same local projects and retained scans as the CLI and MCP
server, so a crawl started in the terminal or by an agent shows up in the window and the other
way round.
Crawler — enter a URL and press Start, no project needed; save the result as a project later.
Project work — goal, tasks, inbox and run history in a With agent display, or the same
projects, scans and reports without AI features in a Simple display.
URLs, issues, link graph, compare — browse retained evidence, rechecks and before/after scans.
Tools — a built-in toolbox: every core tool with a ready-made form, plus a one-click image
pipeline — compress, convert to WebP/AVIF, generate 301 redirects and hand the task to your
developer or agent.
Methods and reports — the core's method skills and report builders behind ready-made screens.
MCP switch — one shared state with the CLI (seohead mcp status); the app registers the
server in Claude Code, Claude Desktop, Codex or Cursor only after a permission dialog that lists
the files and lines it will change, with backups.
Installers — macOS .pkg, Windows setup and Linux tarball put the app and the seohead
command on the machine together. Light, dark and high-contrast themes; English UI.
What it can do
Each area links to the page that documents its method, inputs and limits. The
capability map answers the same question by engineering task.
Area
What you get
Read
Native desktop app
macOS, Windows and Linux: scans, issues, URL inspector, link graph, toolbox
A generated check registry shared by native crawls and SF exports: status codes, redirects, indexability, canonicals, metadata, headings, hreflang, structured data, images, links, sitemaps, rendering and more
--profile full exposes every tool; audit, infra, quick-check and router expose smaller
schema sets for focused sessions. seohead mcp status shows the shared on/off state that the
desktop app uses too. Progress notifications are sent only when the client supplies
a progress token. See MCP profiles and progress.
HTTP-native MCP clients can reach the same registry over streamable HTTP:
seohead mcp --transport http binds 127.0.0.1:8321 by default; stdio stays the default,
and a non-loopback --host is an explicit, warned-about exposure decision
(transports in docs/CLI.md).
An agent discovers routes with seo_tool_catalog, loads a method with seo_skill_show and an
end-to-end chain with seo_scenario_show. A catalogue entry describes a capability; it does not
execute it. The CLI shows the same catalogue:
seohead tool-catalog --query redirect --limit 5
Projects and agent handoff
A project directory keeps the site, agreed scope, crawl policy, checklist coverage, retained scans,
saved finding views and run history, so a new agent continues from the saved project instead of
chat history: define the goal and scope, collect once and analyse again, compare, create developer
tasks, track repairs and report.
seohead watch --project DIRECTORY opens an optional terminal observer beside the chat
(TERMINAL.md). The inbox stores the specialist's notes and proposed goals;
workflow runs checkpoint each registered step with hashed evidence so another agent can resume
exactly where the first stopped (WORKFLOWS.md). Task completion and site
health stay separate: a prepared project or a completed task is not proof that a site error was
fixed. Details: Projects, project-control scenario,
remediation ledger.
What is measured and what is not
Every audit separates four outcomes:
Finding: the available evidence supports a specific problem.
Ran without findings: the check was evaluated and found nothing.
Skipped: required evidence was unavailable or incomplete; the report names the check and reason.
Failed or unavailable tool: the boundary failure is recorded instead of being treated as a pass.
Partial crawls withhold conclusions that need complete evidence, such as link-graph claims, and
health scores are withheld or marked not comparable when coverage is low. Native crawls and
Screaming Frog are different collectors and may discover different URL populations; compare them
only with compatible scope, configuration and provenance.
Known limits: native crawl and browser results are lab evidence, not field Core Web Vitals
(crux-report is credential-gated); site-audit is a bounded sitemap pass, not a link-graph
crawl; backlinks-check verifies donor pages you supply rather than a web-scale index; full
parity with commercial crawlers on very large or JavaScript-heavy sites is a direction, not a
verified claim (COMPARISON.md,
MILLION_CRAWL_ACCEPTANCE.md).
Safety boundaries: network tools block private targets unless explicitly allowed; file mutation,
service-path probes, bot DNS verification, provider production mode and paid calls require
explicit inputs. DataForSEO defaults to sandbox, and paid calls are journalled for spend-report.
Image optimization writes to a separate directory unless in-place mode is requested, which keeps
backups. Secrets and client crawl data never belong in this repository or a client report.
See the audit guideline and GOTCHAS.md.
Repository map
Path
What it holds
seohead/cli/
The seohead command line over the shared handlers
seohead/mcp/
Shared handlers and the local stdio MCP server
seohead/crawl/
Native evidence collection: fetching and parsing, no verdicts
seohead/checks/, seohead/audit/
Audit checks and the audit pipeline over collected evidence
seohead/core/, seohead/storage/
Shared contracts, retained scans and read-only access
seohead/projects/
Portable local project workspaces
seohead/reports/
XLSX, DOCX, CSV, Markdown, JSON and PDF output
seohead/recon/, seohead/data_sources/
Domain/infrastructure recon and external providers
seohead/sf/
Screaming Frog export analysis
seohead/skills/, seohead/data/
Method skills and versioned reference data
seohead/tui/, seohead/integrations/
Optional terminal shell, remote job API and other integrations
desktop/
SpiderHead Desktop (native Qt adapter, MIT; Qt runtime under LGPL v3) with its own toolchain and tests
docs/
Documentation, scenarios, examples and images
tests/
Offline test suite for the core
Development
python -m pip install -e ".[dev,mcp,cluster,reports]"
ruff check .
ruff format --check .
pytest -q
seohead sf run --exports-dir docs/examples/exports --out /tmp/seohead-report --tasks
python -m build
Public commands, counts and generated references are checked in CI: a command shown in the docs
must still run, and generated pages must match the registries. Keep public prose in English, use
synthetic examples and reserved domains only, and add a focused offline test whenever behaviour
changes. See CONTRIBUTING.md, SECURITY.md,
architecture and testing.
Licence and provenance
The core toolkit, documentation and SpiderHead Desktop are released under the MIT License
(desktop/LICENSE carries the same text). The bundled Qt runtime (PySide6/Qt 6) remains under
LGPL v3 on its own terms — see the notices and Qt source offer in
desktop/THIRD_PARTY_NOTICES.md. The bundled Schema.org vocabulary keeps its original
CC BY-SA terms. See
THIRD_PARTY_NOTICES.md, PROVENANCE.md,
TRADEMARKS.md and CITATION.cff.
Screaming Frog and SEO Spider are trademarks of Screaming Frog Ltd. SpiderHead is independent and
not affiliated with or endorsed by Screaming Frog Ltd; the name is used only to describe compatibility.
S
Spiderhead
PavloSEO·4 estrelas·
SpiderHead — SEO crawler & audit toolkit: local-first CLI, MCP server for AI agents and a desktop app. Retained scans, evidence-backed issues, Screaming Frog import.
Ainda não há lista de ferramentas. Servidores remotos recebem uma após a primeira verificação de saúde.
Perguntas frequentes
Perguntas
Como instalo Spiderhead?
Copie nesta página a configuração stdio para Cursor, Claude Desktop, Claude Code ou VS Code. Ela usa a instalação do publicador, não um proxy do PlugMCP.
Spiderhead precisa de uma chave de API?
Este registro não lista nenhuma variável de ambiente obrigatória.
Qual transporte Spiderhead usa?
Spiderhead roda localmente via stdio.
Quando Spiderhead foi verificado pela última vez?
Última verificação: 2026-10-27. SpiderHead — SEO crawler & audit toolkit: local-first CLI, MCP server for AI agents and a desktop app. Retained scans, evidence-backed issues, Screaming Frog import.