Your personal operating system for career transition.

Build a Reference-Based BLEU/ROUGE Scorer

Reference-based metrics compare generated text to a known-good reference. They are the cheapest automated evaluation when you have ground-truth answers.

What to build

1. rouge1_f1(prediction: str, reference: str) -> float — Tokenize by whitespace+lowercase. Compute precision, recall, and F1 on token set overlap. Return F1 (0.0-1.0).

2. bleu1(prediction: str, reference: str) -> float — Unigram BLEU: fraction of prediction tokens that appear in the reference, multiplied by a brevity penalty min(1.0, exp(1 - len(ref_tokens) / len(pred_tokens))) when prediction is shorter.

3. score_dataset(cases: list[dict]) -> dict — Each case has "prediction" and "reference". Return {"avg_rouge1_f1": float, "avg_bleu1": float, "n": int}.

Constraints

• Standard library only (no nltk, no evaluate).
• Empty strings: return 0.0 without crashing.