Invent a Dataset
Generate a 100-row medical Q&A dataset from a single prompt, download it, and score its diversity with DCScore.
This runbook generates a training-ready dataset from scratch with Invent a Dataset. You write one dataset_prompt and need no source data. The example builds a 100-row medical question-answer dataset, then measures how varied the rows are.
Read more on the blog: Invent a Dataset: Data Generation for Custom Models and Invent a Dataset: Measuring Dataset Generation Abilities with Zero Seed Data.
Before you start
Section titled “Before you start”You need an Adaption API key. See Create API keys. In Colab, add it to the secrets panel as ADAPTION_API_KEY. Locally, set it as an environment variable.
Step 1 | Install the SDK
Section titled “Step 1 | Install the SDK”pip install adaptionStep 2 | Create the client
Section titled “Step 2 | Create the client”import os
from adaption import Adaption
if "ADAPTION_API_KEY" not in os.environ: try: from google.colab import userdata # Colab secrets
os.environ["ADAPTION_API_KEY"] = userdata.get("ADAPTION_API_KEY") except ImportError: from getpass import getpass
os.environ["ADAPTION_API_KEY"] = getpass("ADAPTION_API_KEY: ")
client = Adaption()Step 3 | Estimate the cost
Section titled “Step 3 | Estimate the cost”Ask for an estimate first. Passing estimate=True returns the credit cost without starting a run.
estimate = client.datasets.invent( dataset_prompt=( "A dataset of medical question-answer pairs covering diagnoses, treatments, " "drug explanations, and healthcare roles." ), rows=100, estimate=True,)print(f"Estimated credits: {estimate.estimated_credits} | Available credits: {estimate.available_credits}")Step 4 | Invent the dataset
Section titled “Step 4 | Invent the dataset”Drop estimate=True and give the dataset a name. training_type="instruction_dataset" produces prompt and completion pairs. To invent preference pairs for alignment, pass training_type="preference_pairs" instead.
dataset = client.datasets.invent( name="Clinical Q&A", dataset_prompt=( "A dataset of medical question-answer pairs covering diagnoses, treatments, " "drug explanations, and healthcare roles." ), rows=100, training_type="instruction_dataset",)dataset_id = dataset.idprint(dataset_id, dataset.status) # runningInvent can also produce the dataset in other languages and locales. See Expand across languages and locales.
Step 5 | Wait for the run to finish
Section titled “Step 5 | Wait for the run to finish”The run is asynchronous. wait_for_completion polls until the dataset is ready.
record = client.datasets.wait_for_completion(dataset_id)print(f"Generated {record.row_count} rows")Step 6 | Download and view the rows
Section titled “Step 6 | Download and view the rows”import pandas as pd
client.datasets.download( dataset_id, file_format="jsonl",).write_to_file("clinical-qa.jsonl")
df = pd.read_json("clinical-qa.jsonl", lines=True)df.head()In Colab, google.colab.data_table.DataTable(df) gives a sortable, searchable table.
Step 7 | Score diversity with DCScore
Section titled “Step 7 | Score diversity with DCScore”DCScore measures dataset diversity by treating it as a classification task over embeddings. The score runs from 1 (every row identical) to N (every row distinct), so the example also reports it divided by N.
Fetch the reference code and the embedding library. Rows are embedded with all-mpnet-base-v2, the default text model in the DCScore paper.
git clone -q https://github.com/BlueWhaleLab/DCScorepip install -q sentence-transformersimport jsonimport sys
sys.path.append("DCScore")from dcscore_function import DCScore
def evaluate_diversity(texts, model="sentence-transformers/all-mpnet-base-v2", batch_size=128, tau=0.1): """Return DCScore for a list of texts. 1 = all identical, len(texts) = all distinct.""" evaluator = DCScore(model) embeddings, n, _ = evaluator.get_embedding(texts, batch_size=batch_size) score = evaluator.calculate_dcscore_by_embedding(embeddings, kernel_type="cs", tau=tau) return {"dcscore": score, "normalized": score / n, "n": n}
rows = []with open("clinical-qa.jsonl", encoding="utf-8") as file: for line in file: if line.strip(): rows.append(json.loads(line))
data = [f"{r['enhanced_prompt']}\n\n{r['enhanced_completion']}" for r in rows]result = evaluate_diversity(data)print(f"Diversity score: {result['dcscore']:.1f} / {result['n']} rows (normalized {result['normalized']:.3f})")Next steps
Section titled “Next steps”- Grow the dataset with Expand a dataset.
- Check quality with Evaluate dataset quality.
- Train a model on it with Train a model with AutoScientist.