crawl_website

POST

Upload the content of a publicly accessible website for ingestion into a GroundX bucket. This is done by following links within a specified URL, recursively, up to a specified depth or number of pages. Note: this endpoint is currently not supported for on-prem deployments.

Request

This endpoint expects an object.

websiteslist of objectsRequired

Response

Website successfully queued

ingestobject

Errors

get_processing_status_by_id

Up Next

Built with

1	from groundx import GroundX, WebsiteSource
2
3	client = GroundX(
4	api_key="YOUR_API_KEY",
5	)
6	client.documents.crawl_website(
7	websites=[
8	WebsiteSource(
9	bucket_id=1234,
10	cap=10,
11	depth=2,
12	search_data={"key": "value"},
13	source_url="https://my.website.com",
14	)
15	],
16	)