Expand a Dataset
Upload 500 GSM8K rows, grow them with domain, general, and similar rows, and compare what each kind of row looks like.
This runbook takes 500 rows of GSM8K and grows them three ways with Expand data: domain, general, and similar rows, 100 of each. It then shows a sample of each kind so you can compare them.
Runtime is about an hour. Cost is low (3 × 100 added rows); the exact figure is printed before the run.
Before you start
Section titled “Before you start”You need an Adaption API key. See Create API keys. In Colab, add it to the secrets panel as ADAPTION_API_KEY. Locally, set it as an environment variable.
Step 1 | Install and set up
Section titled “Step 1 | Install and set up”pip install adaption pandas pyarrowimport os
from adaption import Adaption
if "ADAPTION_API_KEY" not in os.environ: try: from google.colab import userdata # Colab secrets
os.environ["ADAPTION_API_KEY"] = userdata.get("ADAPTION_API_KEY") except ImportError: from getpass import getpass
os.environ["ADAPTION_API_KEY"] = getpass("ADAPTION_API_KEY: ")
client = Adaption()
PARQUET_URL = "https://huggingface.co/datasets/openai/gsm8k/resolve/main/main/train-00000-of-00001.parquet"NUM_ROWS = 500 # rows of GSM8K to upload; this is the dataset we expand
# One expansion per strategy, so each output dataset holds the seed plus one kind of row.EXPANSIONS = { "domain": {"domain_rows": 100}, "general": {"general_rows": 100}, "similar": {"similar_rows": 100},}
SAMPLES_TO_SHOW = 3Step 2 | Add helper functions
Section titled “Step 2 | Add helper functions”waitwaits for a dataset job and raises with the API’s error message if it failed.downloaddownloads a dataset as JSONL and loads it into a DataFrame.show_rowsshows a sample of rows as prompt and completion cards.added_rowspicks out the rows an expansion added.
import html
import pandas as pdfrom IPython.display import HTML, display
def wait(dataset_id: str, what: str): """Wait for a dataset job and raise with the API's message if it failed.""" status = client.datasets.wait_for_completion(dataset_id, timeout=3 * 3600) if status.status == "failed": msg = (status.error_data and status.error_data.message) or "unknown error" raise RuntimeError(f"{what} failed: {msg}") print(f"{what}: {status.status}, {status.row_count} rows") return status
def download(dataset_id: str, path: str) -> pd.DataFrame: client.datasets.download(dataset_id, file_format="jsonl").write_to_file(path) return pd.read_json(path, lines=True)
def first_present(row, columns): for col in columns: value = row.get(col) if isinstance(value, str) and value.strip() and value.strip() != "N/A": return value return ""
PROMPT_COLS = ["original_prompt", "enhanced_prompt", "prompt"]COMPLETION_COLS = ["original_completion", "enhanced_completion", "completion"]
def show_rows(df: pd.DataFrame, title: str, n: int = SAMPLES_TO_SHOW, seed: int = 0): """Render n rows as prompt/completion cards.""" sample = df.sample(min(n, len(df)), random_state=seed) cards = [] for _, row in sample.iterrows(): prompt = html.escape(first_present(row, PROMPT_COLS)) completion = html.escape(first_present(row, COMPLETION_COLS)) cards.append( "<div style='border:1px solid #d4d4d8;border-radius:8px;padding:12px 14px;margin:10px 0'>" f"<div style='font-weight:600;margin-bottom:4px'>Prompt</div>" f"<div style='white-space:pre-wrap;margin-bottom:10px'>{prompt}</div>" f"<div style='font-weight:600;margin-bottom:4px'>Completion</div>" f"<div style='white-space:pre-wrap;color:#52525b'>{completion}</div>" "</div>" ) display(HTML(f"<h4>{html.escape(title)}</h4>" + "".join(cards)))
def added_rows(df: pd.DataFrame, seed_prompts: set) -> pd.DataFrame: """Rows the expansion added, as opposed to the seed rows it carries over.
The download does not tag rows by origin, so an added row is one whose prompt is not among the seed prompts. """ prompts = df.apply(lambda r: first_present(r, PROMPT_COLS), axis=1) return df[~prompts.isin(seed_prompts)]Step 3 | Take a 500-row slice of GSM8K
Section titled “Step 3 | Take a 500-row slice of GSM8K”GSM8K has about 7,500 grade-school math word problems, each with a step-by-step solution. Download the train split and keep 500 random rows.
import urllib.request
urllib.request.urlretrieve(PARQUET_URL, "gsm8k-train.parquet")seed_df = pd.read_parquet("gsm8k-train.parquet").sample(NUM_ROWS, random_state=0)seed_df.to_parquet("gsm8k-seed.parquet", index=False)print(f"{len(seed_df)} rows, columns: {list(seed_df.columns)}")Step 4 | Upload the rows
Section titled “Step 4 | Upload the rows”Expansion needs a dataset that is ready to read. Instead of running Adaptive Data on it, upload the rows with processing_mode="raw", which takes them as they are: question becomes the prompt and answer the completion. See datasets.upload_file.
upload = client.datasets.upload_file( "gsm8k-seed.parquet", name="GSM8K seed (500 rows)", processing_mode="raw", column_mapping={"prompt": "question", "completion": "answer"},)dataset_id = upload.dataset_idprint(dataset_id)
wait(dataset_id, "Upload")
seed_view = seed_df.rename(columns={"question": "prompt", "answer": "completion"})seed_prompts = set(seed_view["prompt"])show_rows(seed_view, "Seed rows (GSM8K, as uploaded)")Step 5 | Expand with domain, general, and similar rows
Section titled “Step 5 | Expand with domain, general, and similar rows”datasets.augment returns a new dataset with the seed rows plus the added ones. The seed is left unchanged. It supports three kinds of row:
| Strategy | Parameter | Where rows come from |
|---|---|---|
| Domain | domain_rows | Rows matching the topics of your dataset |
| General | general_rows | Rows from other topics, to broaden coverage |
| Similar | similar_rows | New rows in the style, language, and format of your own rows |
You can combine all three in one call. Here, each strategy runs on its own so each output holds one kind of added row. Price all three first:
total = 0for name, rows in EXPANSIONS.items(): est = client.datasets.augment(dataset_id, **rows, estimate=True) total += est.estimated_credits_consumed print(f"{name:>8}: {est.estimated_credits_consumed} credits")print(f"{'total':>8}: {total} credits")Launch all three. The idempotency_key makes re-running safe: a repeat returns the dataset already created instead of starting, and charging for, a second run.
expanded_ids = {}for name, rows in EXPANSIONS.items(): response = client.datasets.augment( dataset_id, **rows, idempotency_key=f"expand-runbook-{dataset_id}-{name}", ) expanded_ids[name] = response.dataset_id print(f"{name:>8}: {response.dataset_id} ({response.status})")Wait for each expansion. A timeout only stops waiting, not the job; run the code again to resume.
for name, expanded_id in expanded_ids.items(): wait(expanded_id, f"{name} expansion")
expanded = {}for name, expanded_id in expanded_ids.items(): df = download(expanded_id, f"gsm8k-{name}.jsonl") new = added_rows(df, seed_prompts) expanded[name] = new print(f"{name:>8}: {len(df)} rows total = {len(df) - len(new)} seed + {len(new)} added")Step 6 | Compare each kind of row
Section titled “Step 6 | Compare each kind of row”The expanded dataset does not label which rows were added, so the code above splits each one by prompt: any row whose prompt is not in the seed was added. Because each expansion ran one strategy, every added row in it comes from that strategy.
Domain rows: more of the same topic
Section titled “Domain rows: more of the same topic”For GSM8K, domain rows are more math, arithmetic, and multi-step word problems. Use them when your dataset is too small to train on but the pool already covers your domain.
show_rows(expanded["domain"], "Domain rows")General rows: broader coverage
Section titled “General rows: broader coverage”General rows come from topics outside your dataset, such as coding, writing, science, and everyday questions. They don’t teach the target task. They keep a fine-tuned model from over-specializing and losing its general abilities. A common recipe keeps about 40% general rows alongside your domain data.
show_rows(expanded["general"], "General rows")Similar rows: written in your style
Section titled “Similar rows: written in your style”Similar rows follow the topic, language, and format of your own rows. Here that means a short word problem, a worked solution with <<…>> calculator notes, and a final #### <answer> line. Use them when the pool has little that looks like your data, or when the format itself matters.
show_rows(expanded["similar"], "Similar rows")Next steps
Section titled “Next steps”- Join the seed and the expansions you want into one training set with Combine datasets, or request all three kinds in one
augmentcall. - Add other languages with
translateandlocalize. - Train on the result with Train a model with AutoScientist.