🕷️ LookaCrawler
Free, open-source, token-efficient local alternative to Firecrawl with native Model Context Protocol (MCP) Server for LLMs.
Усильте агента
Установка от издателя. Не эндпоинт PlugMCP.
Host a copy with our keys and logs.
В реестре пока нет пакетного релиза. Следуйте README репозитория.
Удалённый URL или команда на вашем компьютере? Чем различаются два способа установки
Запустить на PlugMCP
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
ЗапускWeb crawling for Large Language Models (LLMs) is broken by default: modern web pages contain massive HTML bloat (scripts, tracking pixels, nested divs, navigation headers, stylesheets), costing thousands of wasted tokens per page.
LookaCrawler is an open-source, token-optimized local crawler that cuts a median 90% of tokens on real pages (21–98% across 8 sites, see benchmark-live.ts), extracts clean Markdown, renders JavaScript-heavy pages with a real browser, and exposes a native Model Context Protocol (MCP) Server ready for Claude Desktop, Cursor, and Antigravity.
link_format: Choose between inline (standard markdown), references (footnote citations [1], saving ~25% tokens on repetitive URLs), or strip (pure text).image_mode: Choose between ignore (zero tokens), alt_only (preserves semantic context without URL bloat), or markdown (full ).map_website: Inspects /robots.txt, sitemaps, and root anchors to discover all pages in a domain.crawl_website: Breadth-first autonomous crawling with max depth, max pages, URL pattern filters (substring, * wildcard), and real-time token accounting.fast: Ultra-fast native HTTP GET with backoff. Escalates to deep when a page needs JavaScript to render.deep: Headless Playwright engine launching real Google Chrome for pages that need JavaScript. Challenge pages (CAPTCHA, bot checks) are detected and reported instead of retried.extract_web_content, crawl_website, map_website, batch_extract_web_content, extract_structured_data, extract_contacts, search_places, discover_local_leads, deep_enrich_lead, lint_anti_slop, lookup_social_profile, search_social_intent, audit_company_website, analyze_competitors, generate_growth_plan, gtm_pipeline_run.crawler://metrics and cache analytics at crawler://cache/stats.crawl-and-summarize and compare-pages.contacts pulls emails, phone numbers, WhatsApp links and social profiles; places returns structured Google Maps business records with formula-injection-safe CSV output.maps (Google Maps engine) plus enrich, which crawls a company's own site for verified emails, socials and decision makers.audit (SEO + conversion checklist), intel (audit + Maps competitor review + deterministic growth plan), gtm (Maps discovery, enrichment, outreach draft written to JSONL/CSV; never sends anything), antislop (AI-slop copy linter), intent (pt-BR/EN buying-intent scorer), social-lookup and intent-search (public profile meta tags, Reddit .json), daemon (Maps scraper web-mode service).crawler_cache.sqlite to eliminate duplicate network calls.og:title, og:description), publication dates, canonical URLs, and custom CSS selectors.Add LookaCrawler to your claude_desktop_config.json or Cursor MCP settings:
{
"mcpServers": {
"lookacrawler": {
"command": "bun",
"args": ["run", "/absolute/path/to/lookacrawler/index.ts"]
}
}
}
Now you can prompt Claude or Cursor:
"Crawl https://example.com/docs and extract the API documentation using LookaCrawler."
Requires Bun 1.1+ (high-performance runtime with native SQLite):
# Clone the repository
git clone https://github.com/lucasmartins-ai/lookacrawler.git
cd lookacrawler
# Install dependencies
bun install
# Single URL fast Markdown extraction
bun run cli.ts extract https://news.ycombinator.com --mode fast
# Single URL fast Markdown extraction with reference footnotes
bun run cli.ts extract https://example.com --link-format references --output page.md
# Headless Playwright deep extraction with CSS selector target
bun run cli.ts extract https://example.com --mode deep --selector "main" --json
# Discover all website URLs and sitemaps
bun run cli.ts map https://example.com --max-urls 500
# Recursively crawl documentation with URL pattern filtering and token accounting
bun run cli.ts crawl https://example.com/docs --max-depth 2 --max-pages 15 --include /docs/ --link-format references
# Batch concurrent multi-URL crawling
bun run cli.ts batch https://site1.com https://site2.com --concurrency 4
# Structured JSON schema extraction
bun run cli.ts structured https://example.com --schema '{"title":"h1","links":"a"}'
# Extract emails, phones, WhatsApp links and social profiles from a site
bun run cli.ts contacts https://example.com
# Search local businesses on Google Maps (browser-driven)
bun run cli.ts places "dentists in Bristol" --limit 5 --csv -o places.csv
# B2B lead discovery via the bundled google-maps-scraper engine
# (build it first: bun run build:maps)
bun run cli.ts maps "dental clinic" "Bristol" --limit 20
# Enrich a company website: emails, socials, decision makers, short summary
bun run cli.ts enrich https://example.com --title "Example Ltd" --mode fast
# Enrich many leads at once from a JSON array of {title, website}
bun run cli.ts enrich-batch leads.json --concurrency 3 -o enriched.json
# Audit a site: SEO checklist, conversion channels, services, digital maturity
bun run cli.ts audit https://example.com
# Audit + Maps competitor review + deterministic growth plan (--locale en for English)
bun run cli.ts intel --site https://example.com --niche "dental clinic" --loc Bristol --limit 3
# Discover -> enrich -> draft outreach copy -> JSONL (or .csv). Draft only, nothing is sent
bun run cli.ts gtm --niche "dental clinic" --loc Bristol --limit 5 --out leads.jsonl
# Lint copy for AI-slop (argument, --file, or stdin)
bun run cli.ts antislop "No cenário atual, uma solução revolucionária"
# Score buying intent of a post or comment (generic lexicon; --vertical car for vehicles)
bun run cli.ts intent "Procuro um sofá até R$ 2.500"
# Public social lookups, no keys: profile meta tags and Reddit search
bun run cli.ts social-lookup https://www.instagram.com/example/
bun run cli.ts intent-search "recommend a dentist Goiania" --subreddit goiania
# Run the Maps scraper as a local service (127.0.0.1:8088); `maps`, `gtm` and `intel` use it when up
bun run cli.ts daemon start
bun run cli.ts daemon status
bun run cli.ts daemon stop
# Start MCP Server via SSE on port 3000
bun run cli.ts serve --transport sse --port 3000
Behind a TLS-intercepting proxy? Export
LOOKACRAWLER_INSECURE_TLS=trueto accept the proxy's forged certificate. Off by default — never enable it for untrusted networks.
Rotating proxies? Export
LOOKACRAWLER_PROXIESwith a comma or whitespace separated list of URLs. Every request without an explicit--proxypulls the next entry, so one blocked IP is not fatal for the run.
Local proxy (Tor, mitmproxy, a sidecar)? A
proxypassed per request must be a public host. List trusted local proxies inLOOKACRAWLER_ALLOWED_PROXIES(comma separated, exact URLs); entries inLOOKACRAWLER_PROXIESare trusted too.
Every outbound connection goes through a built-in loopback egress guard that
resolves each destination once, refuses private/link-local/loopback addresses
and connects to exactly the IP it checked. This covers redirects and requests a
page makes on its own (fetch, XHR, WebSocket) in deep mode, and prevents DNS
rebinding. With a proxy (--proxy, LOOKACRAWLER_PROXIES) the guard sits in
front of it: it validates the destination, then asks the proxy to connect to
that IP (HTTP CONNECT, SOCKS5 with optional user/password, or SOCKS4), so a
proxy — even a trusted local one — can never be steered at an internal host.
Side effect: DNS is resolved on this machine, not at the proxy, so geo-DNS/CDN
edges follow your location rather than the proxy's, and SOCKS4 cannot reach
IPv6-only sites. LOOKACRAWLER_ALLOW_LOCAL=true turns all of this off and talks
to the proxy directly (for local development only).
The maps command, daemon, gtm, intel and the discover_local_leads MCP
tool drive a Google Maps scraper written in Go. Its source lives in
google-maps-scraper/ (MIT, originally by
Georgios Komninos (gosom), via
coelhobugado/google-maps-scraper,
with local fixes for hangs and leaked browsers).
Responsible use. Google's Maps Platform terms restrict scraping Google Maps content. Use this engine for low-volume research you review yourself, respect each site's terms, and prefer the official Places API for commercial volume. Build it once (needs Go 1.26+;
install-browseralso needs Node.js 18+; the Docker image ships it built):
bun run build:maps
cd google-maps-scraper && ./maps-leads install-browser # first run only
The binary is looked up next to cli.ts, in ../google-maps-scraper/, or at
$MAPS_LEADS_BIN (an explicit path that does not exist is an error, not a
fallback).
Without it, maps fails fast with a clear message; extract, crawl,
contacts, places and enrich are unaffected. Pass --email to let the
scraper extract public emails during discovery.
# Build and run Docker container
docker build -t lookacrawler .
docker run -p 3000:3000 -e LOOKACRAWLER_SSE_TOKEN="$(openssl rand -hex 24)" lookacrawler
The container listens on 0.0.0.0, so a token is always required. Without
LOOKACRAWLER_SSE_TOKEN a random one is generated on each start and printed in
the container logs.
Incoming URL ──► [Local SQLite Cache Check] ──(Hit)──► Return Cached Markdown
│ (Miss)
▼
[Fast HTTP GET Request] ──(Blocked?)──► [Escalate to Browser Rendering]
│ │
▼ ▼
[HTML DOM Tree Parser] ◄──────────────────────────┘
│
▼
[Aggressive Token Noise Pruner]
(Strips SVG, Nav, Ads, Tracking, CSS, JS)
│
▼
[Mozilla Readability Engine]
│
▼
[Turndown Markdown Converter] ──► Return Clean LLM Markdown
Run test suite:
bun test
If LookaCrawler saves you API fees and token costs:
lookacrawler is built and maintained by LookADev, an engineering studio specializing in AI agents, web architecture, and token optimization.
Start a project → lookadev.com · Email: [email protected]
Open-source software licensed under the MIT License.
Free, open-source, token-efficient local alternative to Firecrawl with native MCP Server for LLMs (>73% token reduction).
Инструменты
Списка инструментов пока нет. У удалённых серверов он появляется после первой проверки состояния.
FAQ
Скопируйте на этой странице конфигурацию stdio для Cursor, Claude Desktop, Claude Code или VS Code. Она использует установку издателя, а не прокси PlugMCP.
В этой записи не указано обязательных переменных окружения.
Lookacrawler работает локально через stdio.
Последняя проверка: 2026-11-16. Free, open-source, token-efficient local alternative to Firecrawl with native MCP Server for LLMs (>73% token reduction).