Bulk Uploading in Python
The Python SDK has some quality of life features which make uploading large numbers of documents easier. These chiefly take two forms:
- Ingest Directories: This function will automatically navigate through a document hierarchy and upload compatible documents to a particular GroundX Bucket.
- Ingest Documents: This function allows one to specify a list of local and remotely hosted documents, and handles batching and uploading those documents to GroundX. The
wait_for_complete
flag will cause the function to block the existing python process until the upload process is complete.
Generally, it’s advised to use Ingest Directories if one wants to upload content from a local directory, and Ingest Documents if one wants to upload a large number of individually curated files.