Your personal operating system for career transition.

Build an Eval Result Storage and Comparison Layer

Eval results are only useful if you can compare them across runs. This exercise builds the persistence and comparison layer that makes prompt iteration data-driven.

What to build

Implement EvalStore:

1. save(run_id: str, report: dict) -> None — Persists the report under run_id. Raises ValueError if run_id already exists.

2. load(run_id: str) -> dict — Returns the stored report. Raises KeyError if not found.

3. list_runs() -> list[str] — Returns all run_ids in insertion order.

4. compare(run_id_a: str, run_id_b: str) -> dict — Returns {"run_a", "run_b", "pass_rate_delta", "avg_score_delta", "regression_count_delta", "improved": bool}. improved = True if avg_score_delta > 0 and regression_count_delta <= 0.

5. best_run(metric: str = "avg_score") -> str | None — Returns the run_id with the highest value for the given metric key. Returns None if empty.

Constraints

• Standard library only. In-memory store is fine (no file I/O required).