AI Engineer Portal
Your personal operating system for career transition.
Private mode
Exercise
Create an eval gate for CI/CD pipelines
Implement a EvalGate class that can be used as a quality gate in a CI/CD pipeline:
run(cases: list[dict]) -> EvalGateResult— evaluate all cases and return a result object- Each case has:
id: str,input: str,expected: str,actual: str - A case passes if the actual output contains all key terms from the expected output (use a simple term overlap check: at least 60% of lowercased words in
expectedthat are longer than 3 characters must appear inactual) EvalGateResultshould have:passed: bool,pass_rate: float,failed_cases: list[str](list of case IDs),total: intEvalGatetakes athreshold: float(default 0.85) in its constructor —passedis True ifpass_rate >= threshold- Add a
to_ci_report() -> strmethod onEvalGateResultthat returns a human-readable summary suitable for CI log output
Write a test that demonstrates a gate passing with 9/10 cases and blocking with 8/10 cases (threshold=0.85).
Deployment / medium / Step 4 of 6
Practice stage
General drill
Hint
Keep the solution explicit and reviewable.
Next drill
Build a multi-provider failover chainSuccess criteria
Make the solution explicit, debuggable, and easy to explain.
Review checklist
Review where the boundary is, what gets validated, and what would be hard to debug later.
Practice
Generate a variation
Generate a new exercise variation to deepen understanding or practice a related concept.
Attempt history
Recent submissions
Before you submit, decide what a strong answer should make obvious to the reviewer.
No attempts yet.