crawl_website

POST

https://api.groundx.ai/api/v1/ingest/documents/website

Upload the content of a publicly accessible website for ingestion into a GroundX bucket. This is done by following links within a specified URL, recursively, up to a specified depth or number of pages.

Note1: This endpoint is currently not supported for on-prem deployments. Note2: The source_url must include the protocol, http:// or https://.

Supported Document Types and Ingest Capacities

Authentication

X-API-Keystring

API Key authentication via header

Request

This endpoint expects an object.

websiteslist of objectsRequired

callbackUrlstringOptionalformat: "uri"

The URL that will receive processing event updates.

callbackDatastringOptional

A string that is returned, along with processing event updates, to the callback URL.

Response

Website successfully queued

ingestobject

Errors

400

Bad Request Error

401

Unauthorized Error