AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Implement graceful degradation with fallback tiers

Implement Graceful Degradation with Fallback Tiers

Production AI services fail in partial ways: the primary LLM provider might be rate-limited, the secondary might be slow, and the semantic cache might have a miss. A well-designed system handles these failures without completely breaking user-facing functionality.

The key pattern is explicit degradation tiers — a ranked list of response strategies, from best to worst.

What to build

Implement a DegradationManager that:

Takes an ordered list of ResponseTier objects: name: str, handler: async callable, timeout_seconds: float, is_acceptable: bool.
serve(request: dict) -> DegradedResponse — tries tiers in order, returns the first success. If a tier fails (exception or timeout), log and move on.
DegradedResponse has: content: str, tier_used: str, is_degraded: bool, latency_ms: int, tiers_attempted: list[str].
If all tiers fail, return a DegradedResponse with the static ULTIMATE_FALLBACK_MESSAGE, is_degraded=True.
get_stats() -> dict — return call counts and success rates per tier.

Test with three tiers

Tier 1 (primary LLM): Fast, high quality, is_acceptable=True — but times out on first calls.
Tier 2 (cached): Returns a cached response. is_acceptable=False (degraded).
Tier 3 (template): Returns a static template. is_acceptable=False (degraded).

Constraints

Standard library + asyncio only.
Each tier must respect its timeout_seconds using asyncio.wait_for.

Api Async / hard / Step 12 of 23

Practice stage

Async and provider control

Hint

Make waiting behavior explicit. Timeouts, retries, and concurrency limits matter more than squeezing everything into one helper.

Next drill

Generate Kubernetes manifests for an LLM service

Success criteria

- Uses async boundaries coherently
- Makes timeout and retry decisions legible
- Would be maintainable under provider instability

Review checklist

- Is timeout behavior explicit?
- Is retryable failure separate from terminal failure?
- Would logs reveal what actually timed out?

from __future__ import annotations
import asyncio
import time
from dataclasses import dataclass, field
from typing import Callable, Awaitable

ULTIMATE_FALLBACK_MESSAGE = (
    "Our AI assistant is temporarily unavailable. "
    "Please try again in a few minutes or contact support."
)

@dataclass
class ResponseTier:
    name: str
    handler: Callable[[dict], Awaitable[str]]
    timeout_seconds: float = 10.0
    is_acceptable: bool = True

@dataclass
class DegradedResponse:
    content: str
    tier_used: str
    is_degraded: bool
    latency_ms: int
    tiers_attempted: list[str] = field(default_factory=list)

class DegradationManager:
    def __init__(self, tiers: list[ResponseTier]):
        # TODO: store tiers and initialize stats
        raise NotImplementedError

async def serve(self, request: dict) -> DegradedResponse:
        # TODO: try tiers in order with timeout, return first success
        raise NotImplementedError

def get_stats(self) -> dict:
        # TODO: return per-tier call counts and success rates
        raise NotImplementedError

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.