Evaluate dataset quality

After an adaptation run succeeds, evaluation compares the source and adapted data on quality dimensions measured by the platform. Evaluation runs on its own schedule, so adaptation can finish before quality results are ready.

The Python SDK exposes results in two forms:

• datasets.get_evaluation(dataset_id) returns evaluation status and the full quality result.
• datasets.get(dataset_id) returns the dataset record and its compact evaluation_summary.

Fetch evaluation results

Call get_evaluation with the same dataset_id used for the adaptation run:

evaluation = client.datasets.get_evaluation(dataset_id)

print(evaluation.status)
if evaluation.quality:
    print(f"Score before: {evaluation.quality.score_before}")
    print(f"Score after:  {evaluation.quality.score_after}")
    print(f"Improvement:  {evaluation.quality.improvement_percent}%")

When evaluation succeeds, quality can include before-and-after scores on a 0–10 scale, letter grades, improvement percentage, and an after-adaptation percentile. While status is pending or running, quality can be absent.

Poll until evaluation finishes

Poll evaluation status with a deadline so a notebook or CI job cannot wait forever:

import time

deadline = time.monotonic() + 600

while True:
    evaluation = client.datasets.get_evaluation(dataset_id)
    if evaluation.status in ("succeeded", "failed", "skipped"):
        break
    if time.monotonic() >= deadline:
        raise TimeoutError("Evaluation did not finish within 10 minutes")
    time.sleep(5)

if evaluation.status == "failed":
    raise RuntimeError("Dataset evaluation failed")
if evaluation.status == "skipped":
    print("Evaluation was skipped")
elif evaluation.quality:
    print(evaluation.quality.model_dump(exclude_none=True))

Async clients return the same response shape with await client.datasets.get_evaluation(dataset_id).

Read the summary from the dataset

If your application already fetches the dataset, use evaluation_summary for headline metrics without a separate evaluation request:

dataset = client.datasets.get(dataset_id)

if dataset.evaluation_summary:
    summary = dataset.evaluation_summary
    print(f"Score after: {summary.score_after}")
    print(f"Improvement: {summary.improvement_percent}%")

datasets.get_status reports ingestion and adaptation progress; it does not include evaluation results.

Complete example

This script reads a completed adaptation’s dataset ID from the environment, waits for evaluation, and prints both the detailed result and compact summary:

import os
import time

from adaption import Adaption

client = Adaption()
dataset_id = os.environ["ADAPTION_DATASET_ID"]
deadline = time.monotonic() + 600

while True:
    evaluation = client.datasets.get_evaluation(dataset_id)
    print(f"Evaluation status: {evaluation.status}")

    if evaluation.status in ("succeeded", "failed", "skipped"):
        break
    if time.monotonic() >= deadline:
        raise TimeoutError("Evaluation did not finish within 10 minutes")
    time.sleep(5)

if evaluation.status == "failed":
    raise RuntimeError("Dataset evaluation failed")
if evaluation.status == "skipped":
    print("Evaluation was skipped for this dataset")
elif not evaluation.quality:
    raise RuntimeError("Evaluation succeeded without quality metrics")
else:
    print(evaluation.quality.model_dump(exclude_none=True))

dataset = client.datasets.get(dataset_id)
if dataset.evaluation_summary:
    print(dataset.evaluation_summary.model_dump(exclude_none=True))

Use get_evaluation when you need explicit pipeline status or complete quality details. Use evaluation_summary when a compact result on the dataset record is enough. To iterate economically on quality, limit rows and estimate the run before scaling.