AI Engineer Portal
Your personal operating system for career transition.
Private mode
Exercise
Implement structured request tracing
Implement Structured Request Tracing
Build a span-based tracing system for LLM requests -- the core data collection layer that feeds monitoring dashboards and post-hoc debugging.
What you are building
Implement a Tracer class that:
- Creates a root trace -- with a
trace_id, feature name, and start timestamp. - Supports named spans -- each span has its own start/end timestamps, latency, and attributes.
- Works as a context manager -- spans auto-complete on
__exit__with elapsed time, even if the body raises. - Exports sorted spans --
export()returns a list of dicts sorted bystart_ms.
Usage pattern to support
tracer = Tracer(trace_id="req-123", feature_name="document_qa")
with tracer.span("retrieval", index="docs_v2") as ret_span:
results = retrieve(query)
ret_span.set("num_results", len(results))
with tracer.span("llm_call", model="claude-3-5-sonnet-20241022") as llm_span:
response = call_llm(prompt)
llm_span.set("prompt_tokens", response.usage.input_tokens)
events = tracer.export() # list of dicts, one per span
Constraints
- Use only the Python standard library.
- All spans must capture
span_id,trace_id,name,start_ms,end_ms,latency_ms. - Context manager must capture elapsed time even if the body raises an exception.
Evaluation / medium / Step 10 of 36
Practice stage
Evaluation and review loops
Hint
Separate the scoring logic from the interpretation logic. Your goal is not just a number; it is a useful next action.
Success criteria
- - Produces a useful signal, not decorative output
- - Makes regression review easier
- - Would support a benchmark or observability loop
Review checklist
- - Would this output help decide what to fix next?
- - Are important failure modes visible?
- - Does the score hide any ambiguity I should record?
Related lessons
Practice
Generate a variation
Generate a new exercise variation to deepen understanding or practice a related concept.
Attempt history
Recent submissions
Before you submit, decide what a strong answer should make obvious to the reviewer.
No attempts yet.