方寸 Portal

AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Implement a semantic cache for LLM responses

Implement a SemanticCache class that caches LLM responses keyed by semantic similarity of the query embedding:

  1. set(query: str, embedding: list[float], response: str) -> None — store a response
  2. get(embedding: list[float], threshold: float = 0.92) -> str | None — return the cached response for the most similar query above the threshold, or None
  3. hit_rate() -> float — return the fraction of get() calls that returned a cached result (0.0 if no calls yet)
  4. Use cosine similarity for embedding comparison
  5. The cache should be in-memory (no external dependencies)

Write tests that verify:

  • A query with high similarity (> threshold) returns the cached response
  • A query with low similarity (< threshold) returns None
  • hit_rate() returns the correct fraction after a mix of hits and misses

Deployment / medium / Step 3 of 6

Practice stage

General drill

Hint

Keep the solution explicit and reviewable.

Success criteria

Make the solution explicit, debuggable, and easy to explain.

Review checklist

Review where the boundary is, what gets validated, and what would be hard to debug later.

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.