Crawl Params Preview

1 min read

post/crawl/params-preview

Request example

curl --request POST \
  --url https://api.firecrawl.dev/v2/crawl/params-preview \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "<string>",
  "prompt": "<string>"
}'

Response

{
  "success": true,
  "data": {
    "url": "<string>",
    "includePaths": [
      "<string>"
    ],
    "excludePaths": [
      "<string>"
    ],
    "maxDepth": 1,
    "maxDiscoveryDepth": 1,
    "crawlEntireDomain": true,
    "allowExternalLinks": true,
    "allowSubdomains": true,
    "sitemap": "skip",
    "ignoreQueryParameters": true,
    "ignoreRobotsTxt": true,
    "robotsUserAgent": "<string>",
    "deduplicateSimilarURLs": true,
    "delay": 1,
    "limit": 1
  }
}

post/crawl/params-preview

Authorization

Authorizationstringheaderrequired#

Bearer authentication header of the form Bearer <token>, where <token> is your API key.

Body

application/json

urlstringrequired#

The URL to crawl

promptstringrequired#

Natural language prompt describing what you want to crawl

Length: max 10000

Responses

application/json

Successful response with generated crawl parameters

successboolean#

dataobject#

Show child attributes

urlstring#

The URL to crawl

includePathsstring[]#

URL patterns to include

excludePathsstring[]#

URL patterns to exclude

maxDepthinteger#

Maximum crawl depth

maxDiscoveryDepthinteger#

Maximum discovery depth

crawlEntireDomainboolean#

Whether to crawl the entire domain

allowExternalLinksboolean#

Whether to allow external links

allowSubdomainsboolean#

Whether to allow subdomains

sitemapenum<string>#

Sitemap handling strategy

Options:skipinclude

ignoreQueryParametersboolean#

Whether to ignore query parameters

ignoreRobotsTxtboolean#

Whether robots.txt rules are ignored

robotsUserAgentstring#

Custom User-Agent string used for robots.txt evaluation

deduplicateSimilarURLsboolean#

Whether to deduplicate similar URLs

delaynumber#

Delay between requests in milliseconds

limitinteger#

Maximum number of pages to crawl

Get Crawl Status Cancel Crawl