Running AutoScientist

Create a run

The SDK reads ADAPTION_API_KEY from the environment. Call autoscientist.create with the prepared dataset:

from adaption import Adaption

client = Adaption()

dataset_id = "dataset_abc123"
run = client.autoscientist.create(dataset_id=dataset_id)
print(run.id, run.status)  # pending

Parameter	Required	Notes
dataset_id	Yes	An adapted dataset ID, or a raw dataset.
max_iterations	No	From 1 through 5.
target_win_rate	No	Greater than 0.5 and at most 1. Stops the loop early once reached.
model	No	An ID returned by client.autoscientist.list_models(). If omitted, AutoScientist selects a model for the dataset.
training_method	No	instruction for SFT or alignment for an SFT → DPO pipeline. Pass alignment explicitly for a preference-pair dataset.
hyperparams	No	Overrides to the platform recipe; unset keys retain platform defaults.
augmentation_domain_rows	No	Domain-targeted rows added before training.
augmentation_general_rows	No	General-purpose rows added before training.
idempotency_key	No	Repeated requests with the same key return the existing run.
column_mapping	No	Maps prompt and completion for SFT, or prompt, chosen, and rejected for DPO. Omit it to let the platform infer the mapping.

column_mapping is validated against the adapted dataset’s schema, not the uploaded file’s original headers. Adaptation can add columns such as enhanced_prompt. A mapping to a missing column fails with Selected column '...' for prompt is not in this dataset.

Train on preference pairs

First, complete an Adaptive Data preference-pair run. A preference-pair dataset does not by itself select alignment, so set training_method explicitly:

run = client.autoscientist.create(
    dataset_id=dataset_id,
    training_method="alignment",
)
print(run.id, run.training_method)  # alignment

Standard Adaptive Data output can be inferred without column_mapping. If the adapted schema instead exposes separate custom columns, map all three roles:

run = client.autoscientist.create(
    dataset_id=dataset_id,
    training_method="alignment",
    column_mapping={
        "prompt": "question",
        "chosen": "preferred_answer",
        "rejected": "other_answer",
    },
)

DPO requires at least 12,000 effective preference pairs, including requested augmentation. SFT requires at least 1,000 effective rows. Selecting alignment does not convert another dataset type: the API resolves the run to instruction unless the dataset was prepared as preference_pairs. Read run.training_method to confirm the resolved objective.

AutoScientist presents alignment as one run, but trains an SFT adapter before the DPO stage. The public run ID and resolved training_method="alignment" remain stable across both stages. During the handoff, the run stays running, metrics can be temporarily unavailable, and download_available is false. A cancellation request during this brief handoff can return 409; retry after an iteration becomes active.

Wait for completion

Run status is pending, running, succeeded, failed, or cancelled.

from adaption import TrainingTimeout

try:
    run = client.autoscientist.wait_for_completion(run.id)
except TrainingTimeout as exc:
    print(exc.resource_id, exc.last_status)
    raise

if run.status == "failed":
    raise RuntimeError(f"failed: {run.error}")
if run.status != "succeeded":
    raise RuntimeError(f"Run ended with status: {run.status}")

print(run.iterations_completed, run.max_iterations, run.best_win_rate)

wait_for_completion backs off from 10 to 60 seconds and has a four-hour default timeout. A timeout stops waiting, not training. Call it again with the same run ID to resume tracking.

To control the polling schedule yourself:

import time

while True:
    run = client.autoscientist.get(run.id)
    print(run.status, run.iterations_completed, run.best_win_rate)

    if run.status == "succeeded":
        break
    if run.status == "failed":
        raise RuntimeError(f"failed: {run.error}")
    if run.status == "cancelled":
        raise RuntimeError("Run was cancelled")

    time.sleep(30)

A successful run either reached target_win_rate or used its final iteration. Compare best_win_rate with the target to determine which occurred. Use client.autoscientist.cancel(run.id) to stop a run in flight.

Constrain model size

In the app, Tiny AutoScientist limits selection to models under 10B parameters. With the SDK, review the supported models, list the currently available models, and pass an eligible ID explicitly:

response = client.autoscientist.list_models()
for model in response.models:
    print(model.id)

model_id = input("Eligible model ID: ").strip()
run = client.autoscientist.create(
    dataset_id=dataset_id,
    model=model_id,
)

Model availability and metadata can change. Confirm the current values with autoscientist.list_models rather than inferring model size from its ID.