Your personal operating system for career transition.

Build a Quality Monitoring Dashboard Data Pipeline

Transform raw LLM request traces into hourly quality snapshots suitable for a time-series monitoring dashboard.

What you are building

Implement a QualityPipeline that:

1. Ingests raw trace events -- each event has trace_id, feature_name, timestamp_ms, pass_rate, latency_ms, prompt_tokens, completion_tokens.
2. Aggregates into hourly buckets -- for each (feature, hour) pair, compute: sample count, avg pass rate, P95 latency, total tokens, estimated cost.
3. Detects anomalies -- flags hours where the pass rate is more than 1.5 standard deviations below the feature's 7-day rolling average.
4. Generates a dashboard payload -- returns a dict keyed by feature name with hourly snapshots and an anomalies list.
5. Estimates cost -- use pricing of $3/million input tokens and $15/million output tokens.

Constraints

• Use only the Python standard library.
• Bucket events by UTC hour (floor timestamp to hour boundary).
• Anomaly detection requires at least 7 hourly data points before flagging.