AI observability
For a single request, you should be able to inspect the input, retrieved context, provider response, post-processing result, latency, and cost.
Without that, AI failures turn into anecdotes:
- "the model was weird"
- "retrieval felt off"
- "it timed out somehow"
Observability turns those guesses into engineering work.
Minimum trace shape
For one request, capture:
- request identifier
- user input or task type
- retrieved context or documents
- model/provider used
- output produced
- latency
- token or cost estimate
- failure or review notes
What this enables
Good traces let you answer:
- Did retrieval fail?
- Did the wrong model/config get used?
- Did post-processing break a good response?
- Are latency spikes tied to one dependency?
Trace by layer
For AI systems, observability usually needs to span:
- context assembly
- provider call
- post-processing
- persistence or review handoff
If you only log the final answer, you lose the most valuable debugging information.
Anti-patterns
- logging too little to diagnose failures
- logging everything with no structure
- having traces that humans cannot actually review
Practical takeaway
An AI trace should help you explain both what happened and what to try next.