Extract

1 min read

post/extract

Request example

curl --request POST \
  --url https://api.firecrawl.dev/v1/extract \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "urls": [
    "<string>"
  ],
  "prompt": "<string>",
  "schema": {},
  "enableWebSearch": false,
  "ignoreSitemap": false,
  "includeSubdomains": true,
  "showSources": false,
  "scrapeOptions": {
    "onlyMainContent": true,
    "includeTags": [
      "<string>"
    ],
    "excludeTags": [
      "<string>"
    ],
    "maxAge": 0,
    "headers": {},
    "waitFor": 0,
    "mobile": false,
    "skipTlsVerification": false,
    "timeout": 30000,
    "parsePDF": true,
    "jsonOptions": {
      "schema": {},
      "systemPrompt": "<string>",
      "prompt": "<string>",
      "checkPromptInjection": false
    },
    "actions": [
      {
        "type": "wait",
        "milliseconds": 1,
        "selector": "#my-element"
      }
    ],
    "location": {
      "country": "US",
      "languages": [
        "en-US"
      ]
    },
    "removeBase64Images": true,
    "blockAds": true,
    "proxy": "basic",
    "storeInCache": true,
    "threatProtection": {
      "mode": "off",
      "riskScoreThreshold": 75,
      "blacklist": [
        "<string>"
      ],
      "whitelist": [
        "<string>"
      ],
      "blockedTlds": [
        "<string>"
      ],
      "failurePolicy": "open"
    },
    "formats": [
      "markdown"
    ],
    "changeTrackingOptions": {
      "modes": [
        "git-diff"
      ],
      "schema": {},
      "prompt": "<string>",
      "tag": null
    }
  },
  "ignoreInvalidURLs": false,
  "threatProtection": {
    "mode": "off",
    "riskScoreThreshold": 75,
    "blacklist": [
      "<string>"
    ],
    "whitelist": [
      "<string>"
    ],
    "blockedTlds": [
      "<string>"
    ],
    "failurePolicy": "open"
  }
}'

Response

{
  "success": true,
  "id": "<string>",
  "invalidURLs": [
    "<string>"
  ]
}

post/extract

Note: A new v2 version of this API is now available with improved features and performance.

Authorization

Authorizationstringheaderrequired#

Bearer authentication header of the form Bearer <token>, where <token> is your API key.

Body

application/json

urlsstring[]required#

promptstring#

Prompt to guide the extraction process

schemaobject#

Schema to define the structure of the extracted data. Must conform to JSON Schema.

enableWebSearchboolean#

When true, the extraction will use web search to find additional data

Default: false

ignoreSitemapboolean#

When true, sitemap.xml files will be ignored during website scanning

Default: false

includeSubdomainsboolean#

When true, subdomains of the provided URLs will also be scanned

Default: true

showSourcesboolean#

When true, the sources used to extract the data will be included in the response as sources key

Default: false

scrapeOptionsobject#

Show child attributes

onlyMainContentboolean#

Only return the main content of the page excluding headers, navs, footers, etc.

Default: true

includeTagsstring[]#

Tags to include in the output.

excludeTagsstring[]#

Tags to exclude from the output.

maxAgeinteger#

Returns a cached version of the page if it is younger than this age in milliseconds. If a cached version of the page is older than this value, the page will be scraped. If you do not need extremely fresh data, enabling this can speed up your scrapes by 500%. Defaults to 0, which disables caching.

Default: 0

headersobject#

Headers to send with the request. Can be used to send cookies, user-agent, etc.

waitForinteger#

Specify a delay in milliseconds before fetching the content, allowing the page sufficient time to load.

Default: 0

mobileboolean#

Set to true if you want to emulate scraping from a mobile device. Useful for testing responsive pages and taking mobile screenshots.

Default: false

skipTlsVerificationboolean#

Skip TLS certificate verification when making requests

Default: false

timeoutinteger#

Timeout in milliseconds for the request

Default: 30000

parsePDFboolean#

Controls how PDF files are processed during scraping. When true, the PDF content is extracted and converted to markdown format, with billing based on the number of pages (1 credit per page). When false, the PDF file is returned in base64 encoding with a flat rate of 1 credit total.

Default: true

jsonOptionsobject#

JSON options object

Show child attributes

schemaobject#

The schema to use for the extraction (Optional). Must conform to JSON Schema.

systemPromptstring#

The system prompt to use for the extraction (Optional)

promptstring#

The prompt to use for the extraction without a schema (Optional)

checkPromptInjectionboolean#

When enabled, scans the scraped page content for prompt injection attempts before running the extraction. If an injection is detected, the request fails with a 403 and error code SCRAPE_PROMPT_INJECTION_DETECTED. Adds 4 credits when the check runs. Defaults to false.

Default: false

actionsobject[]#

Actions to perform on the page before grabbing the content

Show child attributes

One of

typeenum<string>required#

Wait for a specified amount of milliseconds

Options:wait

millisecondsinteger#

Number of milliseconds to wait

Range: >= 1

selectorstring#

Query selector to find the element by

typeenum<string>required#

Take a screenshot. The links will be in the response's actions.screenshots array.

Options:screenshot

fullPageboolean#

Whether to capture a full-page screenshot or limit to the current viewport.

Default: false

qualityinteger#

The quality of the screenshot, from 1 to 100. 100 is the highest quality.

typeenum<string>required#

Click on an element

Options:click

selectorstringrequired#

Query selector to find the element by

allboolean#

Clicks all elements matched by the selector, not just the first one. Does not throw an error if no elements match the selector.

Default: false

typeenum<string>required#

Write text into an input field, text area, or contenteditable element. Note: You must first focus the element using a 'click' action before writing. The text will be typed character by character to simulate keyboard input.

Options:write

textstringrequired#

Text to type

typeenum<string>required#

Press a key on the page

Options:press

keystringrequired#

Key to press

typeenum<string>required#

Scroll the page or a specific element

Options:scroll

directionenum<string>#

Direction to scroll

Default: "down"

Options:updown

selectorstring#

Query selector for the element to scroll

typeenum<string>required#

Scrape the current page content, returns the url and the html.

Options:scrape

typeenum<string>required#

Execute JavaScript code on the page

Options:executeJavascript

scriptstringrequired#

JavaScript code to execute

typeenum<string>required#

Generate a PDF of the current page. The PDF will be returned in the actions.pdfs array of the response.

Options:pdf

formatenum<string>#

The page size of the resulting PDF

Default: "Letter"

Options:A0A1A2A3A4A5A6LetterLegalTabloidLedger

landscapeboolean#

Whether to generate the PDF in landscape orientation

Default: false

scalenumber#

The scale multiplier of the resulting PDF

Default: 1

locationobject#

Location settings for the request. When specified, this will use an appropriate proxy if available and emulate the corresponding language and timezone settings. Defaults to 'US' if not specified.

Show child attributes

countrystring#

ISO 3166-1 alpha-2 country code (e.g., 'US', 'AU', 'DE', 'JP')

Default: "US"

Pattern: ^[A-Z]{2}$

languagesstring[]#

Preferred languages and locales for the request in order of priority. Defaults to the language of the specified location. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language

removeBase64Imagesboolean#

Removes all base 64 images from the output, which may be overwhelmingly long. The image's alt text remains in the output, but the URL is replaced with a placeholder.

Default: true

blockAdsboolean#

Enables ad-blocking and cookie popup blocking.

Default: true

proxyenum<string>#

Specifies the type of proxy to use.

basic: Proxies for scraping sites with none to basic anti-bot solutions. Fast and usually works.
enhanced: Enhanced proxies for scraping sites with advanced anti-bot solutions. Slower, but more reliable on certain sites. Billed at the same credit cost as basic.
auto: Firecrawl will automatically retry scraping with enhanced proxies if the basic proxy fails. Enhanced proxies carry no credit surcharge, so either way only the regular cost is billed.

If you do not specify a proxy, Firecrawl will default to basic.

Options:basicenhancedauto

storeInCacheboolean#

If true, the page will be stored in the Firecrawl index and cache. Setting this to false is useful if your scraping activity may have data protection concerns. Using some parameters associated with sensitive scraping (actions, headers) will force this parameter to be false.

Default: true

threatProtectionobject#

Per-request Threat Protection override. Fields you provide replace the corresponding fields of your organization's policy for this request only; omitted fields keep their organization-level values. Requires Threat Protection to be enabled for your team (enterprise feature) — otherwise the request is rejected with a 403. If your organization has disabled request overrides, any request that includes this object is rejected with a 403. If Threat Protection is enforced for your team, mode may not be set to off.

Show child attributes

modeenum<string>#

URL scanning mode for this request. normal checks URLs against Google Web Risk (+2 credits per URL scanned).

Options:offnormal

riskScoreThresholdinteger#

Normalized risk score (0–100) at or above which a classifier verdict blocks the URL. Lower is stricter.

Range: 0–100

blackliststring[]#

Domains to always block, as plain domains (example.com) or wildcard globs (*.example.com). No protocol, path, or port.

Items: max 1000

whiteliststring[]#

Domains to always allow, as plain domains or wildcard globs. Wins over every other rule.

Items: max 1000

blockedTldsstring[]#

Top-level domains to block outright, lowercase without the leading dot (e.g. zip).

Items: max 1000

failurePolicyenum<string>#

What to do when the classifier can't be reached: closed blocks the request, open allows it.

Options:openclosed

formatsenum<string>[]#

Formats to include in the output.

Default: ["markdown"]

changeTrackingOptionsobject#

Options for change tracking (Beta). Only applicable when 'changeTracking' is included in formats. The 'markdown' format must also be specified when using change tracking.

Show child attributes

modesenum<string>[]#

The mode to use for change tracking. 'git-diff' provides a detailed diff, and 'json' compares extracted JSON data.

schemaobject#

Schema for JSON extraction when using 'json' mode. Defines the structure of data to extract and compare. Must conform to JSON Schema.

promptstring#

Prompt to use for change tracking when using 'json' mode. If not provided, the default prompt will be used.

tagstring#

Tag to use for change tracking. Tags can separate change tracking history into separate "branches", where change tracking with a specific tagwill only compare to scrapes made in the same tag. If not provided, the default tag (null) will be used.

Default: null

ignoreInvalidURLsboolean#

If invalid URLs are specified in the urls array, they will be ignored. Instead of them failing the entire request, an extract using the remaining valid URLs will be performed, and the invalid URLs will be returned in the invalidURLs field of the response.

Default: false

threatProtectionobject#

Show child attributes

modeenum<string>#

URL scanning mode for this request. normal checks URLs against Google Web Risk (+2 credits per URL scanned).

Options:offnormal

riskScoreThresholdinteger#

Normalized risk score (0–100) at or above which a classifier verdict blocks the URL. Lower is stricter.

Range: 0–100

blackliststring[]#

Domains to always block, as plain domains (example.com) or wildcard globs (*.example.com). No protocol, path, or port.

Items: max 1000

whiteliststring[]#

Domains to always allow, as plain domains or wildcard globs. Wins over every other rule.

Items: max 1000

blockedTldsstring[]#

Top-level domains to block outright, lowercase without the leading dot (e.g. zip).

Items: max 1000

failurePolicyenum<string>#

What to do when the classifier can't be reached: closed blocks the request, open allows it.

Options:openclosed

Responses

application/json

Successful extraction

successboolean#

idstring#

invalidURLsstring[]#

If ignoreInvalidURLs is true, this is an array containing the invalid URLs that were specified in the request. If there were no invalid URLs, this will be an empty array. If ignoreInvalidURLs is false, this field will be undefined.

Search Get Extract Status