XCrawl
XCrawl is a web search and data crawling tool designed for AI application scenarios, providing core capabilities including web page scraping, sitemap parsing, batch data retrieval, SERP (search engine results page) aggregation, and LLM-oriented search interfaces. The tool is intended to help developers and data analysts efficiently collect internet data, supports multilingual interfaces, and is applicable to AI training data preparation and information retrieval tasks.
Timeline (last 90 days)
The XCrawl MCP server supports passing the API key via the Authorization: Bearer request header, avoiding keys appearing in URLs.
XCrawl's MCP server npm package is named xcrawl-mcp and connects to the XCrawl web data API.
The tools exposed by the XCrawl MCP server include xcrawl_scrape, xcrawl_search, xcrawl_map, xcrawl_crawl, xcrawl_check_crawl_status and xcrawl_key_status.
In a web scraping workflow, you should map to list URLs first and then crawl selectively, to control context budget and noise.
Running the XCrawl MCP server via local stdio requires Node.js 18 or higher.
Newly registered XCrawl accounts are provided with 1,000 free credits.
For MCP clients that cannot set request headers, the key can be written directly into the path: https://mcp.answerline.dev/<key>/mcp.
All Facts (7)
The XCrawl MCP server supports passing the API key via the Authorization: Bearer request header, avoiding keys appearing in URLs.
50%UnverifiedXCrawl's MCP server npm package is named xcrawl-mcp and connects to the XCrawl web data API.
50%UnverifiedThe tools exposed by the XCrawl MCP server include xcrawl_scrape, xcrawl_search, xcrawl_map, xcrawl_crawl, xcrawl_check_crawl_status and xcrawl_key_status.
50%UnverifiedIn a web scraping workflow, you should map to list URLs first and then crawl selectively, to control context budget and noise.
50%UnverifiedRunning the XCrawl MCP server via local stdio requires Node.js 18 or higher.
50%UnverifiedNewly registered XCrawl accounts are provided with 1,000 free credits.
50%UnverifiedFor MCP clients that cannot set request headers, the key can be written directly into the path: https://mcp.answerline.dev/<key>/mcp.
50%