Initiate a batch upload

POST/api/v1/datasets/upload/initiate-batch

Initiate a batch upload. A batch must be homogeneous: either all document formats (pdf, docx, pptx, xlsx, html, zip) or all tabular formats (csv, json, jsonl, parquet). Mixed batches are rejected. TXT files are permissive and may be included in either kind of batch (or alone, in which case the dataset is tabular); their contents are converted during complete-batch.

Body ParametersJSONExpand Collapse

name: string

Human-readable name for the dataset

files: array of object { file_name, file_format, file_size_bytes }

List of files to upload. A single batch must be homogeneous: either all document formats (pdf, docx, pptx, xlsx, html, zip) or all tabular formats (csv, json, jsonl, parquet). Mixing the two is rejected because the two pipelines use different credit models and preprocessing stages. TXT files are permissive: they may be included in either kind of batch (or alone, producing a tabular dataset). complete-batch converts TXT contents to csv/jsonl and merges them into the resulting dataset.

file_name: string

Original file name

file_format: "csv" or "json" or "jsonl" or 8 more

Format of the file

One of the following:

"csv"

"json"

"jsonl"

"parquet"

"pdf"

"docx"

"pptx"

"xlsx"

"html"

"zip"

"txt"

file_size_bytes: number

File size in bytes

defer_adaption: optional boolean

When true, the upload is stored but Adaptive Data does not start. The dataset stays in awaiting_preprocessing until POST /datasets/:id/start-adaption.

ReturnsExpand Collapse

dataset_id: string

Dataset ID

uploads: array of object { file_name, upload_url, s3_key }

file_name: string

Original file name

upload_url: string

Presigned S3 upload URL

s3_key: string

S3 key for the uploaded file

Initiate a batch upload

curl https://api.prod.adaptionlabs.ai/api/v1/datasets/upload/initiate-batch \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $ADAPTION_API_KEY" \
    -d '{
          "name": "my-document-dataset",
          "files": [
            {
              "file_name": "report.pdf",
              "file_format": "pdf",
              "file_size_bytes": 1048576
            }
          ]
        }'

{
  "dataset_id": "dataset_id",
  "uploads": [
    {
      "file_name": "file_name",
      "upload_url": "upload_url",
      "s3_key": "s3_key"
    }
  ]
}

Returns Examples

{
  "dataset_id": "dataset_id",
  "uploads": [
    {
      "file_name": "file_name",
      "upload_url": "upload_url",
      "s3_key": "s3_key"
    }
  ]
}