# Batch Scrape

> Scrape multiple URLs in a single batch job

import BatchScrapePython from '/snippets/v2/batch-scrape/base/python.mdx';
import BatchScrapeNode from '/snippets/v2/batch-scrape/base/js.mdx';
import BatchScrapeCURL from '/snippets/v2/batch-scrape/base/curl.mdx';
import BatchScrapeOutput from '/snippets/v2/batch-scrape/base/output.mdx';
import BatchScrapeAsyncOutput from '/snippets/v2/batch-scrape/base/async-output.mdx';
import BatchScrapeExtractPython from '/snippets/v2/batch-scrape/json/python.mdx';
import BatchScrapeExtractNode from '/snippets/v2/batch-scrape/json/js.mdx';
import BatchScrapeExtractCURL from '/snippets/v2/batch-scrape/json/curl.mdx';
import BatchScrapeExtractOutput from '/snippets/v2/batch-scrape/json/output.mdx';
import BatchScrapeExtractAsyncOutput from '/snippets/v2/batch-scrape/json/async-output.mdx';
import BatchScrapeWebhookCURL from '/snippets/v1/batch-scrape-webhook/base/curl.mdx';

Batch scrape lets you scrape multiple URLs in a single job. Pass a list of URLs and optional parameters, and Firecrawl processes them concurrently and returns all results together.

- Works like `/crawl` but for an explicit list of URLs
- Synchronous and asynchronous modes
- Supports all scrape options including structured extraction
- Configurable concurrency per job

## Operations

| Task | API reference | SDK vocabulary |
|---|---|---|
| Start a batch | [`batch-scrape`](/api-reference/endpoint/batch-scrape) | `batchScrape` / `batch_scrape`, `startBatchScrape` / `start_batch_scrape` |
| Check status and results | [`batch-scrape-get`](/api-reference/endpoint/batch-scrape-get) | `getBatchScrapeStatus` / `get_batch_scrape_status` |
| Cancel a running batch | [`batch-scrape-delete`](/api-reference/endpoint/batch-scrape-delete) | Cancel or delete the batch job by ID |
| Inspect errors | [`batch-scrape-get-errors`](/api-reference/endpoint/batch-scrape-get-errors) | Error/status helper for failed URLs |

## How it works

You can run a batch scrape in two ways:

| Mode | SDK method (JS / Python) | Behavior |
|------|--------------------------|----------|
| Synchronous | `batchScrape` / `batch_scrape` | Starts the batch and waits for completion, returning all results |
| Asynchronous | `startBatchScrape` / `start_batch_scrape` | Starts the batch and returns a job ID for polling or webhooks |

## Basic usage

<CodeGroup>

<BatchScrapePython />
<BatchScrapeNode />
<BatchScrapeCURL />

</CodeGroup>

### Response

Calling `batchScrape` / `batch_scrape` returns the full results when the batch completes.

<BatchScrapeOutput />

Calling `startBatchScrape` / `start_batch_scrape` returns a job ID you can track via `getBatchScrapeStatus` / `get_batch_scrape_status`, the API endpoint `/batch/scrape/{id}`, or webhooks. Job results are available via the API for 24 hours after completion. After this period, you can still view your batch scrape history and results in the [activity logs](https://www.firecrawl.dev/app/logs).

<BatchScrapeAsyncOutput />

## Concurrency

By default, a batch scrape job uses your team's full concurrent browser limit (see [Rate Limits](/rate-limits)). You can lower this per job with the `maxConcurrency` parameter.

For example, `maxConcurrency: 50` limits that job to 50 simultaneous scrapes. Setting this value too low on large batches will significantly slow down processing, so only reduce it if you need to leave capacity for other concurrent jobs.

## Structured extraction

You can use batch scrape to extract structured data from every page in the batch. This is useful when you want the same schema applied to a list of URLs.

<CodeGroup>

<BatchScrapeExtractPython />
<BatchScrapeExtractNode />
<BatchScrapeExtractCURL />

</CodeGroup>

### Response

`batchScrape` / `batch_scrape` returns full results:

<BatchScrapeExtractOutput />

`startBatchScrape` / `start_batch_scrape` returns a job ID:

<BatchScrapeExtractAsyncOutput />

## Webhooks

You can configure webhooks to receive real-time notifications as each URL in your batch is scraped. This lets you process results immediately instead of waiting for the entire batch to complete.

<BatchScrapeWebhookCURL />

### Event types

| Event | Description |
|-------|-------------|
| `batch_scrape.started` | The batch scrape job has begun |
| `batch_scrape.page` | A single URL was successfully scraped |
| `batch_scrape.completed` | All URLs have been processed |
| `batch_scrape.failed` | The batch scrape encountered an error |

### Payload

Each webhook delivery includes a JSON body with the following structure:

```json
{
  "success": true,
  "type": "batch_scrape.page",
  "id": "batch-job-id",
  "data": [...],
  "metadata": {},
  "error": null
}
```

### Verifying webhook signatures

Every webhook request from Firecrawl includes an `X-Firecrawl-Signature` header containing an HMAC-SHA256 signature. Always verify this signature to ensure the webhook is authentic and has not been tampered with.

1. Get your webhook secret from the [Advanced tab](https://www.firecrawl.dev/app/settings?tab=advanced) of your account settings
2. Extract the signature from the `X-Firecrawl-Signature` header
3. Compute HMAC-SHA256 of the raw request body using your secret
4. Compare with the signature header using a timing-safe function

<Warning>
  Never process a webhook without verifying its signature first. The `X-Firecrawl-Signature` header contains the signature in the format: `sha256=abc123def456...`
</Warning>

For complete implementation examples in JavaScript and Python, see the [Webhook Security documentation](/webhooks/security).

For comprehensive webhook documentation including detailed event payloads, advanced configuration, and troubleshooting, see the [Webhooks documentation](/webhooks/overview).

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.