Scrape

1 min read

post/scrape

Request example

curl --request POST \
  --url https://api.firecrawl.dev/v1/scrape \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "<string>",
  "onlyMainContent": true,
  "includeTags": [
    "<string>"
  ],
  "excludeTags": [
    "<string>"
  ],
  "maxAge": 0,
  "headers": {},
  "waitFor": 0,
  "mobile": false,
  "skipTlsVerification": false,
  "timeout": 30000,
  "parsePDF": true,
  "jsonOptions": {
    "schema": {},
    "systemPrompt": "<string>",
    "prompt": "<string>",
    "checkPromptInjection": false
  },
  "actions": [
    {
      "type": "wait",
      "milliseconds": 1,
      "selector": "#my-element"
    }
  ],
  "location": {
    "country": "US",
    "languages": [
      "en-US"
    ]
  },
  "removeBase64Images": true,
  "blockAds": true,
  "proxy": "basic",
  "storeInCache": true,
  "threatProtection": {
    "mode": "off",
    "riskScoreThreshold": 75,
    "blacklist": [
      "<string>"
    ],
    "whitelist": [
      "<string>"
    ],
    "blockedTlds": [
      "<string>"
    ],
    "failurePolicy": "open"
  },
  "formats": [
    "markdown"
  ],
  "changeTrackingOptions": {
    "modes": [
      "git-diff"
    ],
    "schema": {},
    "prompt": "<string>",
    "tag": null
  },
  "zeroDataRetention": false
}'

Response

{
  "success": true,
  "data": {
    "markdown": "<string>",
    "html": "<string>",
    "rawHtml": "<string>",
    "screenshot": "<string>",
    "links": [
      "<string>"
    ],
    "actions": {
      "screenshots": [
        "<string>"
      ],
      "scrapes": [
        {
          "url": "<string>",
          "html": "<string>"
        }
      ],
      "javascriptReturns": [
        {
          "type": "<string>",
          "value": null
        }
      ],
      "pdfs": [
        "<string>"
      ]
    },
    "metadata": {
      "title": "<string>",
      "description": "<string>",
      "language": "<string>",
      "sourceURL": "<string>",
      "keywords": "<string>",
      "ogLocaleAlternate": [
        "<string>"
      ],
      "<any other metadata> ": "<string>",
      "statusCode": 1,
      "numPages": 1,
      "totalPages": 1,
      "error": "<string>"
    },
    "llm_extraction": {},
    "warning": "<string>",
    "changeTracking": {
      "previousScrapeAt": "2026-08-10T12:00:00Z",
      "changeStatus": "new",
      "visibility": "visible",
      "diff": "<string>",
      "json": {}
    }
  }
}

post/scrape

Note: A new v2 version of this API is now available with improved features and performance.

Authorization

Authorizationstringheaderrequired#

Bearer authentication header of the form Bearer <token>, where <token> is your API key.

Body

application/json

urlstringrequired#

The URL to scrape

onlyMainContentboolean#

Only return the main content of the page excluding headers, navs, footers, etc.

Default: true

includeTagsstring[]#

Tags to include in the output.

excludeTagsstring[]#

Tags to exclude from the output.

maxAgeinteger#

Returns a cached version of the page if it is younger than this age in milliseconds. If a cached version of the page is older than this value, the page will be scraped. If you do not need extremely fresh data, enabling this can speed up your scrapes by 500%. Defaults to 0, which disables caching.

Default: 0

headersobject#

Headers to send with the request. Can be used to send cookies, user-agent, etc.

waitForinteger#

Specify a delay in milliseconds before fetching the content, allowing the page sufficient time to load.

Default: 0

mobileboolean#

Set to true if you want to emulate scraping from a mobile device. Useful for testing responsive pages and taking mobile screenshots.

Default: false

skipTlsVerificationboolean#

Skip TLS certificate verification when making requests

Default: false

timeoutinteger#

Timeout in milliseconds for the request

Default: 30000

parsePDFboolean#

Controls how PDF files are processed during scraping. When true, the PDF content is extracted and converted to markdown format, with billing based on the number of pages (1 credit per page). When false, the PDF file is returned in base64 encoding with a flat rate of 1 credit total.

Default: true

jsonOptionsobject#

JSON options object

Show child attributes

schemaobject#

The schema to use for the extraction (Optional). Must conform to JSON Schema.

systemPromptstring#

The system prompt to use for the extraction (Optional)

promptstring#

The prompt to use for the extraction without a schema (Optional)

checkPromptInjectionboolean#

When enabled, scans the scraped page content for prompt injection attempts before running the extraction. If an injection is detected, the request fails with a 403 and error code SCRAPE_PROMPT_INJECTION_DETECTED. Adds 4 credits when the check runs. Defaults to false.

Default: false

actionsobject[]#

Actions to perform on the page before grabbing the content

Show child attributes

One of

typeenum<string>required#

Wait for a specified amount of milliseconds

Options:wait

millisecondsinteger#

Number of milliseconds to wait

Range: >= 1

selectorstring#

Query selector to find the element by

typeenum<string>required#

Take a screenshot. The links will be in the response's actions.screenshots array.

Options:screenshot

fullPageboolean#

Whether to capture a full-page screenshot or limit to the current viewport.

Default: false

qualityinteger#

The quality of the screenshot, from 1 to 100. 100 is the highest quality.

typeenum<string>required#

Click on an element

Options:click

selectorstringrequired#

Query selector to find the element by

allboolean#

Clicks all elements matched by the selector, not just the first one. Does not throw an error if no elements match the selector.

Default: false

typeenum<string>required#

Write text into an input field, text area, or contenteditable element. Note: You must first focus the element using a 'click' action before writing. The text will be typed character by character to simulate keyboard input.

Options:write

textstringrequired#

Text to type

typeenum<string>required#

Press a key on the page

Options:press

keystringrequired#

Key to press

typeenum<string>required#

Scroll the page or a specific element

Options:scroll

directionenum<string>#

Direction to scroll

Default: "down"

Options:updown

selectorstring#

Query selector for the element to scroll

typeenum<string>required#

Scrape the current page content, returns the url and the html.

Options:scrape

typeenum<string>required#

Execute JavaScript code on the page

Options:executeJavascript

scriptstringrequired#

JavaScript code to execute

typeenum<string>required#

Generate a PDF of the current page. The PDF will be returned in the actions.pdfs array of the response.

Options:pdf

formatenum<string>#

The page size of the resulting PDF

Default: "Letter"

Options:A0A1A2A3A4A5A6LetterLegalTabloidLedger

landscapeboolean#

Whether to generate the PDF in landscape orientation

Default: false

scalenumber#

The scale multiplier of the resulting PDF

Default: 1

locationobject#

Location settings for the request. When specified, this will use an appropriate proxy if available and emulate the corresponding language and timezone settings. Defaults to 'US' if not specified.

Show child attributes

countrystring#

ISO 3166-1 alpha-2 country code (e.g., 'US', 'AU', 'DE', 'JP')

Default: "US"

Pattern: ^[A-Z]{2}$

languagesstring[]#

Preferred languages and locales for the request in order of priority. Defaults to the language of the specified location. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language

removeBase64Imagesboolean#

Removes all base 64 images from the output, which may be overwhelmingly long. The image's alt text remains in the output, but the URL is replaced with a placeholder.

Default: true

blockAdsboolean#

Enables ad-blocking and cookie popup blocking.

Default: true

proxyenum<string>#

Specifies the type of proxy to use.

basic: Proxies for scraping sites with none to basic anti-bot solutions. Fast and usually works.
enhanced: Enhanced proxies for scraping sites with advanced anti-bot solutions. Slower, but more reliable on certain sites. Billed at the same credit cost as basic.
auto: Firecrawl will automatically retry scraping with enhanced proxies if the basic proxy fails. Enhanced proxies carry no credit surcharge, so either way only the regular cost is billed.

If you do not specify a proxy, Firecrawl will default to basic.

Options:basicenhancedauto

storeInCacheboolean#

If true, the page will be stored in the Firecrawl index and cache. Setting this to false is useful if your scraping activity may have data protection concerns. Using some parameters associated with sensitive scraping (actions, headers) will force this parameter to be false.

Default: true

threatProtectionobject#

Per-request Threat Protection override. Fields you provide replace the corresponding fields of your organization's policy for this request only; omitted fields keep their organization-level values. Requires Threat Protection to be enabled for your team (enterprise feature) — otherwise the request is rejected with a 403. If your organization has disabled request overrides, any request that includes this object is rejected with a 403. If Threat Protection is enforced for your team, mode may not be set to off.

Show child attributes

modeenum<string>#

URL scanning mode for this request. normal checks URLs against Google Web Risk (+2 credits per URL scanned).

Options:offnormal

riskScoreThresholdinteger#

Normalized risk score (0–100) at or above which a classifier verdict blocks the URL. Lower is stricter.

Range: 0–100

blackliststring[]#

Domains to always block, as plain domains (example.com) or wildcard globs (*.example.com). No protocol, path, or port.

Items: max 1000

whiteliststring[]#

Domains to always allow, as plain domains or wildcard globs. Wins over every other rule.

Items: max 1000

blockedTldsstring[]#

Top-level domains to block outright, lowercase without the leading dot (e.g. zip).

Items: max 1000

failurePolicyenum<string>#

What to do when the classifier can't be reached: closed blocks the request, open allows it.

Options:openclosed

formatsenum<string>[]#

Formats to include in the output.

Default: ["markdown"]

changeTrackingOptionsobject#

Options for change tracking (Beta). Only applicable when 'changeTracking' is included in formats. The 'markdown' format must also be specified when using change tracking.

Show child attributes

modesenum<string>[]#

The mode to use for change tracking. 'git-diff' provides a detailed diff, and 'json' compares extracted JSON data.

schemaobject#

Schema for JSON extraction when using 'json' mode. Defines the structure of data to extract and compare. Must conform to JSON Schema.

promptstring#

Prompt to use for change tracking when using 'json' mode. If not provided, the default prompt will be used.

tagstring#

Tag to use for change tracking. Tags can separate change tracking history into separate "branches", where change tracking with a specific tagwill only compare to scrapes made in the same tag. If not provided, the default tag (null) will be used.

Default: null

zeroDataRetentionboolean#

If true, this will enable zero data retention for this scrape. To enable this feature, please contact help@firecrawl.dev

Default: false

Responses

application/json

Successful response

successboolean#

dataobject#

Show child attributes

markdownstring#

htmlstring#

Cleaned HTML of the page if html is in formats. Removes <script>, <style>, <noscript>, <meta>, and <head> tags; converts relative URLs to absolute; resolves responsive image srcset to the largest version. Respects onlyMainContent, includeTags, and excludeTags filters.

rawHtmlstring#

The exact, unmodified HTML as received from the page if rawHtml is in formats. No cleaning or filtering is applied.

screenshotstring#

Screenshot of the page if screenshot is in formats. Screenshots expire after 24 hours and can no longer be downloaded.

linksstring[]#

List of links on the page if links is in formats

actionsobject#

Results of the actions specified in the actions parameter. Only present if the actions parameter was provided in the request

Show child attributes

screenshotsstring[]#

Screenshot URLs, in the same order as the screenshot actions provided. Screenshots expire after 24 hours and can no longer be downloaded.

scrapesobject[]#

Scrape contents, in the same order as the scrape actions provided.

Show child attributes

urlstring#

htmlstring#

javascriptReturnsobject[]#

JavaScript return values, in the same order as the executeJavascript actions provided.

Show child attributes

typestring#

valueany#

pdfsstring[]#

PDFs generated, in the same order as the pdf actions provided.

metadataobject#

Show child attributes

titlestring#

descriptionstring#

languagestring#

sourceURLstring#

keywordsstring · string[]#

Keywords extracted from the page, can be a string or array of strings

Show child attributes

One of

ogLocaleAlternatestring[]#

Alternative locales for the page

<any other metadata> string#

statusCodeinteger#

The status code of the page

numPagesinteger#

For PDF inputs, the number of pages parsed (capped by the parsers maxPages option).

totalPagesinteger#

For PDF inputs, the document's true page count before any maxPages capping. Omitted when it cannot be determined; a totalPages greater than numPages indicates the result was truncated.

errorstring#

The error message of the page

llm_extractionobject#

Displayed when using LLM Extraction. Extracted data from the page following the schema defined.

warningstring#

Can be displayed when using LLM Extraction. Warning message will let you know any issues with the extraction.

changeTrackingobject#

Change tracking information if changeTracking is in formats. Only present when the changeTracking format is requested.

Show child attributes

previousScrapeAtstring#

The timestamp of the previous scrape that the current page is being compared against. Null if no previous scrape exists.

changeStatusenum<string>#

The result of the comparison between the two page versions. 'new' means this page did not exist before, 'same' means content has not changed, 'changed' means content has changed, 'removed' means the page was removed.

Options:newsamechangedremoved

visibilityenum<string>#

The visibility of the current page/URL. 'visible' means the URL was discovered through an organic route (links or sitemap), 'hidden' means the URL was discovered through memory from previous crawls.

Options:visiblehidden

diffstring#

Git-style diff of changes when using 'git-diff' mode. Only present when the mode is set to 'git-diff'.

jsonobject#

JSON comparison results when using 'json' mode. Only present when the mode is set to 'json'. This will emit a list of all the keys and their values from the previous and current scrapes based on the type defined in the schema. Example here

Introduction Batch Scrape