crawlbase_scrape
Pack: crawlbase · Endpoint: https://gateway.pipeworx.io/crawlbase/mcp
Scrape any website through Crawlbase rotating residential proxies with server-side anti-bot bypass (Cloudflare, DataDome, hCaptcha solved for you). Returns the page as HTML (default), clean markdown (format:“md” — great for LLM context), or a JSON envelope with metadata (format:“json”). JavaScript-heavy pages (SPAs) render when you pass your Crawlbase JavaScript token as _apiKey; the Normal token does fast static fetches. Example: crawlbase_scrape({ url: “https://example.com”, format: “md”, _apiKey: “your-crawlbase-token” })
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
url | string | yes | The absolute URL to scrape including scheme, e.g. “https://example.com/page” |
javascript | boolean | no | Set true when the page needs JavaScript rendering (SPAs, lazy-loaded feeds). Crawlbase selects rendering by TOKEN TYPE, so when this is true you must pass your Crawlbase JavaScript token as _apiKey (the Normal token returns the static HTML shell). Default false. |
country | string | no | Two-letter ISO country code to route the crawl through, e.g. “US”, “GB”, “DE”, “JP”. Default: automatic geo selection. |
format | string | no | Response shape: “html” (default, raw page), “md” (GitHub-flavored markdown, LLM-friendly), or “json” (page + metadata in one JSON envelope). |
device | string | no | Device profile to emulate: “desktop” (default), “tablet”, or “mobile”. |
page_wait | number | no | Milliseconds to wait after page load before capturing (JavaScript token required). Useful for content that animates in. |
ajax_wait | boolean | no | Wait until the network is idle before capturing (JavaScript token required). Best for SPAs that fetch data after mount. |
_apiKey | string | yes | Your Crawlbase token — Normal token for static pages, JavaScript token for rendered pages. Free tier (10,000 requests) at https://crawlbase.com |
Example call
curl -X POST https://gateway.pipeworx.io/crawlbase/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"crawlbase_scrape","arguments":{}}}'Connect
Add this to your MCP client config, or use one-click install buttons:
{
"mcpServers": {
"crawlbase": {
"url": "https://gateway.pipeworx.io/crawlbase/mcp"
}
}
}See Getting Started for client-specific install steps.