Substack publications, posts, subscriber counts, and keyword search.
A Model Context Protocol (MCP) server for performing automated accessibility scans of web pages using Playwright and Axe-core
Rowmeld: merge Apify datasets and remove duplicate rows with smart matching; new rows only.
Any link as llms.txt; check llms.txt; see how AI crawlers read a page; draft AGENTS.md.
Scrape webpages, run crawls, and read Markdown and other results with Nstdata Crawl.
MCPScraper CLI: live web pages as clean Markdown, text or HTML in your terminal, through the MCPScraper API. The commands follow the live API spec.
Checks if GPTBot, ClaudeBot, PerplexityBot, Googlebot and 9 more AI crawlers can reach a page.
Prompt injection scanner for documents, web pages and RAG content: instruction overrides, spoofed chat roles, hidden text and data-exfiltration attempts, as a CLI and an MCP server
58 scrapers via Apify MCP: Booking, Airbnb, Amazon, Google, Zillow, Indeed, Yelp, Reddit, Vinted
Web scraping, Llama 3 digests and Twitter intelligence, paid per call in USDC via x402 on Base.
Firefox automation via WebDriver BiDi: snapshot-based interactions, network monitoring, console capture and screenshots for testing and scraping.
Native-speed, browserless web crawler + MCP server for AI agents. A standalone Rust engine (turbo-dom + a V8 JS-render tier) — fetch + parse + extract + run page JS with no headless browser. This npm
Lighthouse-style CLI for auditing websites for AI agents, LLM crawlers, MCP clients, llms.txt, WebMCP, OpenAPI, and Schema.org readiness
Crawl a website into clean Markdown for LLMs/RAG: main content only, linked docs converted.
Audit how AI sees a website: per-bot robots.txt verdicts for AI crawlers, schema, sitemap, llms.txt, SPA detection and cross-model brand mentions.
Local-first web scraping, DuckDuckGo search, site mapping and crawling, using trafilatura with an optional Firecrawl fallback for JS-rendered pages.
Web scraping via ScraperAPI with JavaScript rendering, geotargeting, premium proxies and auto-parsing.
Identity and credential gateway for agents: Ed25519 identity, encrypted credential vault, OPA policies with audit logging, per-agent memory and web scraping.
URL fetcher and HTML-to-Markdown converter with layered prompt injection defense: hidden-element sanitization, risk scanning and content boundary wrapping.
Use Apify Actors, pre-built cloud tools, to extract data from websites, e-commerce, social media, search engines, maps and more.
Fetch public web pages live and get clean Markdown, text or HTML.
Business emails from websites, UK & EU contract award winners, website tech stacks. Apify Actors.
AgentScraper CLI: live web pages as clean Markdown, text or HTML in your terminal, through the AgentScraper API. The commands follow the live API spec.
Free, open-source, local-first SEO crawler and regression audit tool.
Live Web Data CLI: live web pages as clean Markdown, text or HTML in your terminal, through the Live Web Data API. The commands follow the live API spec.
FetchMCP CLI: live web pages as clean Markdown, text or HTML in your terminal, through the FetchMCP API. The commands follow the live API spec.
Content licensing for the AI era. Licensed, rights-cleared content for AI agents, with a verifiable license key for every article. The MCP alternative to unlicensed scraping for RAG and AI search.
MCP server for Google Play and App Store scraping
AI-powered web scraping MCP. Distill pages to Markdown or extract structured data via JSON Schema.
Google Web Search: Google Web Search API. Search the world’s information, including webpages.
Read public web content with Evercraft; search and browser automation stay behind safety gates.
Apify MCP — run web-scraping Actors and fetch their dataset results.
foundit (Monster India) jobs as JSON: any function, full JD, salary, apply link. $0.80/1k
Gesamtes Verzeichnis
Das öffentliche Web crawlen, extrahieren und überwachen. 1,399 Server, die meistgenutzten zuerst. Jeder öffnet die Seite des Herausgebers.