方寸 Portal

AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Implement structured request tracing

Implement Structured Request Tracing

Build a span-based tracing system for LLM requests -- the core data collection layer that feeds monitoring dashboards and post-hoc debugging.

What you are building

Implement a Tracer class that:

  1. Creates a root trace -- with a trace_id, feature name, and start timestamp.
  2. Supports named spans -- each span has its own start/end timestamps, latency, and attributes.
  3. Works as a context manager -- spans auto-complete on __exit__ with elapsed time, even if the body raises.
  4. Exports sorted spans -- export() returns a list of dicts sorted by start_ms.

Usage pattern to support

tracer = Tracer(trace_id="req-123", feature_name="document_qa")

with tracer.span("retrieval", index="docs_v2") as ret_span:
    results = retrieve(query)
    ret_span.set("num_results", len(results))

with tracer.span("llm_call", model="claude-3-5-sonnet-20241022") as llm_span:
    response = call_llm(prompt)
    llm_span.set("prompt_tokens", response.usage.input_tokens)

events = tracer.export()  # list of dicts, one per span

Constraints

  • Use only the Python standard library.
  • All spans must capture span_id, trace_id, name, start_ms, end_ms, latency_ms.
  • Context manager must capture elapsed time even if the body raises an exception.

Evaluation / medium / Step 10 of 36

Practice stage

Evaluation and review loops

Hint

Separate the scoring logic from the interpretation logic. Your goal is not just a number; it is a useful next action.

Success criteria
  • - Produces a useful signal, not decorative output
  • - Makes regression review easier
  • - Would support a benchmark or observability loop
Review checklist
  • - Would this output help decide what to fix next?
  • - Are important failure modes visible?
  • - Does the score hide any ambiguity I should record?

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.