commoncrawl_crawls

Pack: commoncrawl · Endpoint: https://gateway.pipeworx.io/commoncrawl/mcp

No MCP client? Call it directly: GET https://gateway.pipeworx.io/v1/tools/commoncrawl_crawls for the schema, then POST the same URL with its arguments for the data.

List the Common Crawl monthly crawl collections (crawl ids like “CC-MAIN-2026-34”) with the date range each one covers. AUTHORITATIVE for “which Common Crawl snapshot covers ” — call this first to pick the crawl id that commoncrawl_index_search needs. Newest first. Keyless.

Parameters

NameTypeRequiredDescription
limitnumbernoHow many crawls to return, newest first. Default 20, max 120 (the full history back to 2008).
contains_datestringnoOptional ISO date, e.g. “2025-01-15”. Returns only the crawl(s) whose capture window covers that date — the answer to “which snapshot would have seen my page then”.

Example call

Arguments

{
  "limit": 3
}

curl

curl -X POST https://gateway.pipeworx.io/commoncrawl/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"commoncrawl_crawls","arguments":{"limit":3}}}'

TypeScript (@pipeworx/sdk)

import { Pipeworx } from '@pipeworx/sdk';
const pipeworx = new Pipeworx();

const result = await pipeworx.call('commoncrawl_crawls', {
  "limit": 3
});

More examples

{
  "contains_date": "2025-01-15",
  "limit": 5
}

Connect

Add this to your MCP client config, or use one-click install buttons:

{
  "mcpServers": {
    "commoncrawl": {
      "url": "https://gateway.pipeworx.io/commoncrawl/mcp"
    }
  }
}

See Getting Started for client-specific install steps.

Regenerated from source · build September 18, 2026