Skip to main content

Get Crawl Status

1 min read
get/crawl/{id}
Request example
Response
get/crawl/{id}

Authorization

Authorizationstringheaderrequired#
Bearer authentication header of the form Bearer <token>, where <token> is your API key.

Path parameters

idstringrequired#
The ID of the crawl job

Responses

application/json
Successful response
statusstring#
The current status of the crawl. Can be scraping, completed, or failed.
totalinteger#
The total number of pages that were attempted to be crawled.
completedinteger#
The number of pages that have been successfully crawled.
creditsUsedinteger#
The number of credits used for the crawl.
expiresAtstring#
The date and time when the crawl will expire.
nextstring#
The URL to retrieve the next 10MB of data. Returned if the crawl is not completed or if the response is larger than 10MB.
dataobject[]#
The data of the crawl.
Show child attributes
markdownstring#
htmlstring#
HTML version of the content on page if includeHtml is true
rawHtmlstring#
Raw HTML content of the page if includeRawHtml is true
screenshotstring#
Screenshot of the page if includeScreenshot is true
metadataobject#
Show child attributes
titlestring · string[]#
Title extracted from the page, can be a string or array of strings
Show child attributes
descriptionstring · string[]#
Description extracted from the page, can be a string or array of strings
Show child attributes
languagestring · string[]#
Language extracted from the page, can be a string or array of strings
Show child attributes
sourceURLstring#
keywordsstring · string[]#
Keywords extracted from the page, can be a string or array of strings
Show child attributes
ogLocaleAlternatestring[]#
Alternative locales for the page
<any other metadata> string · string[]#
Other metadata extracted from HTML, can be a string or array of strings
Show child attributes
statusCodeinteger#
The status code of the page
numPagesinteger#
For PDF inputs, the number of pages parsed (capped by the parsers maxPages option).
totalPagesinteger#
For PDF inputs, the document's true page count before any maxPages capping. Omitted when it cannot be determined; a totalPages greater than numPages indicates the result was truncated.
errorstring#
The error message of the page