AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Implement model routing by task complexity

Implement Model Routing by Task Complexity

Routing every request to your most powerful model is expensive. Routing every request to your cheapest model produces weak results on complex tasks. The right engineering move is a routing layer that matches model capability to task complexity automatically.

What you are building

Build a ModelRouter class that:

Maintains a list of ModelTier configs, each with a model name, a cost weight, and capability flags
Implements a route method that accepts a RoutingRequest and returns the most appropriate ModelTier
Routes based on measurable signals:
- Input length in tokens (longer inputs need more capable models)
- Task type (code, reasoning, extraction, summarization, chat)
- User tier (free, pro, enterprise)
Respects explicit overrides (if force_model is set, use it)
Falls back to the cheapest capable model if no tier fully matches

The signal rules (implement these)

code or reasoning tasks → prefer Sonnet-tier or above
free user tier → cap at Haiku-tier
Input tokens > 8000 → require a model with supports_long_context = True
All other tasks → Haiku-tier is sufficient

Why this matters

Model routing is one of the highest-leverage cost optimizations in a real LLM application. Typical production systems route 70–90% of requests to cheap models and reserve expensive models for complex tasks. The savings at scale are significant.

Llm Foundations / medium / Step 3 of 6

Practice stage

General drill

Hint

Keep the solution explicit and reviewable.

Next drill

Build a token cost tracker

Success criteria

Make the solution explicit, debuggable, and easy to explain.

Review checklist

Review where the boundary is, what gets validated, and what would be hard to debug later.

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum
from typing import Optional

class TaskType(str, Enum):
    code = "code"
    reasoning = "reasoning"
    extraction = "extraction"
    summarization = "summarization"
    chat = "chat"

class UserTier(str, Enum):
    free = "free"
    pro = "pro"
    enterprise = "enterprise"

@dataclass
class ModelTier:
    name: str                         # e.g. "claude-haiku-4-5"
    tier_label: str                   # "haiku", "sonnet", "opus"
    cost_weight: float                # relative cost (1.0 = baseline)
    supports_long_context: bool = False
    max_output_tokens: int = 4096

@dataclass
class RoutingRequest:
    task_type: TaskType
    user_tier: UserTier
    estimated_input_tokens: int
    force_model: Optional[str] = None

@dataclass
class RoutingDecision:
    model: ModelTier
    reason: str

class ModelRouter:
    """Route LLM requests to the most cost-efficient capable model."""

def __init__(self, tiers: list[ModelTier]) -> None:
        # TODO: store tiers sorted by cost_weight ascending
        raise NotImplementedError

def route(self, request: RoutingRequest) -> RoutingDecision:
        """
        Select the appropriate model tier for this request.

Rules:
        1. If force_model is set, use that model (raise ValueError if not found)
        2. Free users get capped at haiku-tier
        3. code or reasoning tasks require sonnet-tier or higher
        4. Inputs > 8000 tokens require supports_long_context = True
        5. Otherwise use the cheapest capable model
        """
        # TODO: implement routing logic
        raise NotImplementedError

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.