Why this matters

Prompt changes are the most frequent and least-tested change in AI systems. A developer tweaks the system prompt to improve one behavior, ships it, and three other behaviors quietly regress. Without a regression test suite, you find out from users. With one, you find out in CI before the change reaches production.

Prompt regression testing is the eval discipline that closes this loop.

Core concepts

What prompt regression means

A prompt regression is any change to a prompt — system prompt, user message template, few-shot examples, tool descriptions — that causes previously-correct behavior to become incorrect. The behavior being measured is always relative to a baseline: the golden set.

A golden set is a collection of (input, expected output) pairs that represent correct behavior at a known-good point in time. When the prompt changes, you re-run the inputs and compare new outputs to the golden expected outputs.

Golden dataset design

A good golden set:

• Covers the behavioral surface you care about (not just happy paths)
• Has explicit acceptance criteria for each case, not just free-form notes
• Is version controlled alongside the prompt
• Is small enough to run in CI (under 5 minutes) but large enough to catch regressions (30-100 cases is typical)

# golden_set.jsonl — one JSON object per line
# {"id": "...", "input": {...}, "expected": "...", "min_score": 0.8, "tags": [...]}

import json
from pathlib import Path


def load_golden_set(path: str) -> list[dict]:
    return [json.loads(line) for line in Path(path).read_text().splitlines() if line.strip()]

Building the regression runner

import difflib
from dataclasses import dataclass


@dataclass
class RegressionResult:
    case_id: str
    input: dict
    expected: str
    actual: str
    score: float
    passed: bool
    diff: str


def similarity_score(expected: str, actual: str) -> float:
    """Token overlap similarity, range 0.0-1.0."""
    ratio = difflib.SequenceMatcher(None, expected.lower(), actual.lower()).ratio()
    return round(ratio, 4)


def run_regression(
    system_fn,
    golden_set: list[dict],
    threshold: float = 0.8,
) -> tuple[list[RegressionResult], bool]:
    results = []
    all_passed = True

    for case in golden_set:
        actual = system_fn(**case["input"])
        expected = case["expected"]
        score = similarity_score(expected, actual)
        passed = score >= case.get("min_score", threshold)
        if not passed:
            all_passed = False
        diff = "
".join(difflib.unified_diff(
            expected.splitlines(), actual.splitlines(),
            fromfile="expected", tofile="actual", lineterm=""
        ))
        results.append(RegressionResult(case["id"], case["input"], expected, actual, score, passed, diff))

    return results, all_passed

Automated regression detection in CI

# run_regression_ci.py — called by CI, exits non-zero on regression
import sys
import json


def main():
    golden = load_golden_set("evals/golden_set.jsonl")
    results, all_passed = run_regression(my_system_function, golden)

    # Write report
    report = {
        "total": len(results),
        "passed": sum(1 for r in results if r.passed),
        "failed": sum(1 for r in results if not r.passed),
        "cases": [vars(r) for r in results],
    }
    Path("eval_report.json").write_text(json.dumps(report, indent=2))

    # Print failures to stderr so CI surfaces them
    for r in results:
        if not r.passed:
            print(f"REGRESSION: {r.case_id} score={r.score}
{r.diff}", file=sys.stderr)

    sys.exit(0 if all_passed else 1)


if __name__ == "__main__":
    main()

In GitHub Actions:

- name: Run regression tests
  run: python run_regression_ci.py
  # Job fails automatically if exit code is non-zero

When to update the golden set

Update the golden set when you intentionally change behavior. The process:

1. Make the prompt change
2. Run regression — note which cases fail
3. Review each failing case: is the new output actually better?
4. If yes, update the golden expected value and add a comment explaining why
5. Commit the updated golden set alongside the prompt change

This makes regressions visible and deliberate rather than silent.

Common mistakes

Using exact-match only — LLM outputs are rarely byte-for-byte identical across runs. Use semantic similarity or a rubric-based scorer, not exact string equality, unless the output space is truly constrained.

Golden set rot — not updating the golden set when behavior legitimately improves. After six months the set becomes a source of false alarms that developers learn to ignore. Review and prune it quarterly.

Too few cases — 5 golden cases catch nothing. 30-50 cases across behavioral dimensions (happy path, edge cases, adversarial inputs, format requirements) provide meaningful signal.

Running regressions only on main — regression tests are most valuable before merging. Run them on pull requests so regressions are caught before review, not after.

Try it yourself

Create a golden set of 10 cases for a summarization or classification function you have built or can build quickly. Each case should have an id, input, expected output, and minimum score. Build the regression runner above, run it against the current implementation to establish a baseline (all should pass), then deliberately modify the system prompt in a way that breaks at least two cases. Verify the runner exits non-zero and surfaces the diffs. Finally, update the golden set for one of the broken cases to reflect the new (intentionally different) behavior and confirm the runner passes again.