Skip to content
Adaption Adaption Adaption
SupportLogin
Introduction
Adaptive Data quickstart

Adaptive Data quickstart

Install the Adaption Python SDK, import data, run Adaptive Data, and download the adapted dataset.

Install the SDK

Section titled “Install the SDK”
Terminal window
pip install adaption

Create an API key

Section titled “Create an API key”
  1. Sign in to Adaption.

  2. Open Settings, then open API keys.

  3. Create a key and copy it. You will not be able to view the key again.

Use the key in Python

Section titled “Use the key in Python”

For local testing, you can pass the key directly:

from adaption import Adaption


client = Adaption(api_key="pt_live_...")

For applications and CI, set ADAPTION_API_KEY:

Terminal window
export ADAPTION_API_KEY="pt_live_..."

The SDK reads the environment variable automatically:

from adaption import Adaption


client = Adaption()

Create a dataset

Section titled “Create a dataset”

This quickstart imports the Alpaca dataset from Hugging Face:

dataset = client.datasets.create(
    source={
        "url": "https://huggingface.co/datasets/tatsu-lab/alpaca",
        "files": [
            "data/train-00000-of-00001-a09b74b3ef9c3b56.parquet"
        ],
    },
)
dataset_id = dataset.dataset_id
print(dataset_id)

You can upload a local file or import from Kaggle instead. Each source uses the same datasets.create endpoint and returns the dataset_id used by the rest of this guide.

Upload a local file

Create the dataset, PUT its bytes to the returned presigned URL, then confirm the upload.

Supported extensions: .csv, .json, .jsonl, .parquet, .pdf, .docx, .pptx, .xlsx, .html, .zip, and .txt.

import hashlib
from pathlib import Path


import httpx


path = Path("training_data.csv")
data = path.read_bytes()


dataset = client.datasets.create(
    source={
        "name": path.name,
        "file_format": "csv",
    },
)
response = httpx.put(dataset.upload_instructions.url, content=data)
response.raise_for_status()


client.datasets.upload.complete_by_id(
    dataset.dataset_id,
    file_size_bytes=len(data),
    sha256=hashlib.sha256(data).hexdigest(),
)
dataset_id = dataset.dataset_id
print(dataset_id)
Import from Kaggle

Use the Kaggle dataset page URL and the files to pull.

dataset = client.datasets.create(
    source={
        "url": "https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset",
        "files": ["spam.csv"],
    },
)
dataset_id = dataset.dataset_id
print(dataset_id)

Kaggle credentials must be registered in Adaption: open API keys settings (sign in if prompted) and add your Kaggle API credentials there before importing.

Imports run asynchronously. Wait until ingestion has populated the dataset before starting Adaptive Data.

Wait for the dataset to import

Section titled “Wait for the dataset to import”

Poll the dataset status until the imported row count is available:

import time


while True:
    status = client.datasets.get_status(dataset_id)
    if status.status == "failed":
        err = status.error_data
        msg = (err and err.message) or "unknown error"
        raise RuntimeError(f"Ingestion failed: {msg}")
    if status.row_count is not None:
        print(f"Imported {status.row_count} rows")
        break
    time.sleep(5)

Adapt the dataset

Section titled “Adapt the dataset”

Adapt applies the platform’s recipes and optimizations. Map the source columns to the roles Adaptive Data expects. For Alpaca, instruction is the prompt, output is the completion, and input is context.

Request an estimate first. This validates the configuration and returns a quote without starting the run:

column_mapping = {
    "prompt": "instruction",
    "completion": "output",
    "context": ["input"],
}
job_specification = {"max_rows": 5_000}


estimate = client.datasets.run(
    dataset_id,
    column_mapping=column_mapping,
    job_specification=job_specification,
    estimate=True,
)
print(f"Estimated credits: {estimate.estimated_credits_consumed}")

When you are ready to proceed, submit the same configuration without estimate=True:

run = client.datasets.run(
    dataset_id,
    column_mapping=column_mapping,
    job_specification=job_specification,
)
print(f"Run started: {run.run_id}")

Wait for Adaptive Data completion

Section titled “Wait for Adaptive Data completion”

The SDK helper polls the dataset status until the adaptation run finishes:

status = client.datasets.wait_for_completion(dataset_id, timeout=3600)
if status.status == "failed":
    raise RuntimeError(f"Adaptive Data failed: {status.error_data}")


print(status.status)

Download the adapted dataset

Section titled “Download the adapted dataset”

Export augmented data for training pipelines, evaluation harnesses, or downstream storage. download streams the processed rows in the requested file format. It accepts a file_format argument of csv, json, jsonl, or parquet, and defaults to csv.

client.datasets.download(dataset_id, file_format="parquet").write_to_file(
    "adapted-alpaca.tar.gz"
)

download returns the file itself, not a link to it. Read the bytes with .read(), write them straight to disk with .write_to_file(path), or decode a text format with .text() — a method, not a property. This shape requires adaption >= 0.9.0; earlier releases returned a string and corrupted every binary format.

For large exports, stream instead of holding the body in memory:

with client.datasets.with_streaming_response.download(
    dataset_id, file_format="parquet"
) as export:
    export.stream_to_file("adapted-alpaca.tar.gz")

file_format="parquet" returns a gzipped tar archive of Parquet shards, not a single .parquet file — extract it before reading:

import tarfile


with tarfile.open("adapted-alpaca.tar.gz") as archive:
    archive.extractall("adapted-alpaca/")

A dataset with status succeeded contains the full output. For a failed dataset, the download contains rows processed before the run stopped; the API returns 422 only if no run has started.

Going further

Section titled “Going further”