Crawl a website for links, file download links, HTML tables as rows, and lists, with no AI
260 MCP tools across 49 domains. AI Flywheel, quality gates, research, web scraping.
Web Tools and Web Tasks for AI agents. 100-credit MCP trial; API-key credits or x402.
Asks 8 AI crawlers for your page, checks who really answered, and writes the fixes. MIT.
Scrape Bluesky posts, profiles, followers, threads and keyword search. Clean JSON, pay per result.
Apify actor via Apify's MCP server (Apify token needed): Greenhouse, Lever, Ashby jobs, one schema.
Apify actor via Apify's MCP server (Apify token needed): Whisper transcription, per-file status.
Apify actor via Apify's MCP server (Apify token needed): bulk OCR with per-page status.
Apify actor via Apify's MCP server (Apify token needed): ChatGPT search brand visibility tracker.
Apify actor via Apify's MCP server (Apify token needed): full-page website screenshots and PDFs.
Capture batches of public web pages in a real browser and download the screenshots as a zip.
official mcp server for crawlbrulee — EU-native web scraping for AI agents & developers. native scrape and map tools for your agent.
Web pages to markdown, site crawls, company contacts, app reviews, ATS jobs and Shopify products.
Hosts AI-generated web content (HTML, images, SVG, JSON) at a public URL, with custom domains.
Finds the robots.txt lines that reserve nothing — retired crawler tokens, shadowed groups, rules abo
Google Play apps, reviews and search, plus top-100 search rank on Google, Bing and Yandex.
Website to clean Markdown or JSON for AI and RAG: read pages, crawl sites, extract described fields.
Check what AI crawlers may read and what a page permits, generate llms.txt and schema.
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
Convert PDF, Word, Excel, PowerPoint, e-mails and web pages to clean Markdown for LLMs.
Reddit posts and comments by keyword, subreddit, or thread URL, via Arctic Shift or your search key
Transcripts of TikTok videos or whole accounts, with views, likes and dates.
Fetch and extract web pages, discover links, and read llms.txt documentation indexes.
E-commerce competitor intelligence, VOC sentiment, Amazon ASIN analysis, Reddit growth playbooks.
Does an AI crawler see your site? Free registry lookup, x402 pricing; checks paid in USDC on Base.
Live Google for Jobs postings: salary, apply links, remote and date filters. Pay per job.
Scraping API for Google Search, Maps, Flights, Jobs, YouTube, LinkedIn, Trustpilot and more.
AI search readiness MCP: score a site for ChatGPT/Claude/Perplexity crawlers. 3 free/day.
Apify MCP — run web-scraping Actors and fetch their dataset results.
Parse a User-Agent: browser, OS, device, plus bot, search crawler, AI crawler and AI agent flags.
HTML string or https URL to markdown. x402 $0.005 USDC, Base. As is, no warranty.
Greenhouse/Lever jobs via Apify MCP; needs your Apify token. Unofficial; Ondalith is AI-operated.
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
Google Web Search: Google Web Search API. Search the world’s information, including webpages.
ไดเรกทอรีทั้งหมด
ครอว์ล ดึงข้อมูล และมอนิเตอร์เว็บสาธารณะ 1,395 เซิร์ฟเวอร์ เรียงจากที่ใช้มากที่สุด แต่ละรายการจะเปิดหน้าของผู้เผยแพร่