AEO tools for AI agents: crawler permissions, llms.txt, structured data, AI-readiness audit.
YouTube channels, playlists, and search lists as flat rows, with no login
Web scraping, inference, DNS, monitoring from residential IPs — pay per request in USDC
EU tenders from TED with competition data: bids per buyer, single-bid rate, winners. Via Apify.
Public website health scan, AI-crawler and structured-data readiness, and a local presence score.
Find agent skills that render videos, decks, images and web pages, and get their install command.
Domain reputation over time: Domain Rating, Tranco rank, TLS, and AI crawler policy.
Web scraping and AI sales workspace with scoped CRM, scraping, knowledge and sales workflow tools.
Screen names/IDs against Ukraine's NSDC State Sanctions Register (snapshot) via Apify.
Due diligence on Canadian companies: registry status, sanctions and risk signals via Apify.
Arachne MCP — scraping, browser, vision, OCR, RAG, Social Research, GitHub + VRT (56 tools).
Search or monitor any self-hosted Cachet status page — not tied to one vendor.
Check a page, AI-crawler access, llms.txt and backlinks of a site. Free, read-only.
Open-source web scraper for AI agents with scrape, crawl, and map tools
MCP server for web scraping: URLs, links, metadata, CSS extraction.
Publish web pages to spacesheep.dev from a terminal or CI: `npx spacesheep deploy ./dist`
Weibo posts, hot search, comments and profiles to JSON. No login or cookies needed.
Is this page readable for AI search? Crawler access, structure and citability checks, runs locally.
MCP stdio server for Polish public law and registers. Lists 17 Apify Actor tools without a token and runs them with APIFY_TOKEN.
Apple App Store reviews as structured JSON via the Apify Reviews API Actor, hosted MCP.
superagnt: marketing plugin for Claude and ChatGPT. Know your market: scrape and search the web.
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
Crawl a website for links, file download links, HTML tables as rows, and lists, with no AI
Capture batches of public web pages in a real browser and download the screenshots as a zip.
Finds the robots.txt lines that reserve nothing — retired crawler tokens, shadowed groups, rules abo
Check what AI crawlers may read and what a page permits, generate llms.txt and schema.
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
Get Shopify product prices, variants and stock availability for competitor monitoring and exports.
Convert PDF, Word, Excel, PowerPoint, e-mails and web pages to clean Markdown for LLMs.
24 public-data Apify actors (sanctions, SEC Form D, WARN, FMCSA, SEO, jobs) as MCP tools.
One key to 1,000+ paid data APIs: enrichment, SEO/SERP, scraping, places, news. Pay per call.
Google Jobs listings billed per result, via the Apify Google Jobs Scraper API, hosted MCP.
Google Maps routes, ETAs, and turn-by-turn steps via the Apify Directions API, hosted MCP.
Google News headlines, sources, and links via the Apify Google News API, hosted MCP.
Reddit posts and comments by keyword, subreddit, or thread URL, via Arctic Shift or your search key
Does an AI crawler see your site? Free registry lookup, x402 pricing; checks paid in USDC on Base.
Tüm dizin
Açık web'i tarama, veri çıkarma ve izleme. 1,395 sunucu, en çok kullanılanlar önce. Her biri yayıncının sayfasını açar.