Skip to content
Adaption Adaption Adaption
SupportLogin
API Reference
Datasets
Custom Evals

List custom-rubric evaluations for a dataset, newest first

datasets.custom_evals.list(strdataset_id) -> CustomEvalListResponse
GET/api/v1/datasets/{dataset_id}/custom-evals

List custom-rubric evaluations for a dataset, newest first

ParametersExpand Collapse
dataset_id: str
ReturnsExpand Collapse
class CustomEvalListResponse: …
items: List[CustomEval]
custom_eval_id: str
dataset_id: str
status: str

pending | running | succeeded | failed

judge_prompt: str
name: Optional[str]
score_before: Optional[float]

Mean rubric score on the original pairs. Null until the eval succeeds.

score_after: Optional[float]

Mean rubric score on the adapted pairs. Null until the eval succeeds.

improvement_percent: Optional[float]

Percentage change from score_before to score_after. Null until the eval succeeds.

scored_rows: Optional[int]

Rows the judge scored. A rubric score is an average over this sample, not the whole dataset.

error_message: Optional[str]
created_at: datetime
formatdate-time
completed_at: Optional[datetime]
formatdate-time

List custom-rubric evaluations for a dataset, newest first

import os
from adaption import Adaption

client = Adaption(
    api_key=os.environ.get("ADAPTION_API_KEY"),  # This is the default and can be omitted
)
custom_evals = client.datasets.custom_evals.list(
    "dataset_id",
)
print(custom_evals.items)
{
  "items": [
    {
      "custom_eval_id": "custom_eval_id",
      "dataset_id": "dataset_id",
      "status": "running",
      "judge_prompt": "judge_prompt",
      "name": "name",
      "score_before": 0,
      "score_after": 0,
      "improvement_percent": 0,
      "scored_rows": 0,
      "error_message": "error_message",
      "created_at": "2019-12-27T18:11:19.117Z",
      "completed_at": "2019-12-27T18:11:19.117Z"
    }
  ]
}
Returns Examples
{
  "items": [
    {
      "custom_eval_id": "custom_eval_id",
      "dataset_id": "dataset_id",
      "status": "running",
      "judge_prompt": "judge_prompt",
      "name": "name",
      "score_before": 0,
      "score_after": 0,
      "improvement_percent": 0,
      "scored_rows": 0,
      "error_message": "error_message",
      "created_at": "2019-12-27T18:11:19.117Z",
      "completed_at": "2019-12-27T18:11:19.117Z"
    }
  ]
}