AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Implement batch processing with configurable rate limiting

Implement Batch Processing with Configurable Rate Limiting

Production AI systems routinely send thousands of requests to provider APIs. Without rate limiting, you hit 429 errors, waste time on retries, and destabilize your provider relationship. The right abstraction is a batch processor that enforces a requests-per-minute budget, retries transient failures with backoff, and reports per-item outcomes without aborting the run.

What to build

Implement a synchronous RateLimitedBatcher class that:

__init__(rpm_limit: int, max_retries: int = 3) — configure requests per minute and retry budget.
process(items: list[dict], fn: Callable[[dict], dict]) -> BatchResult — process all items, calling fn(item) for each, with:
- At most rpm_limit calls per 60 seconds (use time.sleep to enforce the rate window).
- Retry fn(item) up to max_retries times on RetryableError exceptions, with 1-second base backoff doubling each attempt.
- Do NOT retry on FatalError exceptions — record the failure immediately.
- Return a BatchResult with successes: list[dict], failures: list[dict], retry_count: int.
RetryableError and FatalError — custom exception classes (no logic needed, just pass).
BatchResult — a dataclass with successes, failures, and retry_count.

Rate limiting logic

The simplest correct implementation: track the timestamp of each call. If you are about to exceed rpm_limit calls in the last 60 seconds, sleep until the oldest call in the window is more than 60 seconds ago.

Why this matters

Uncontrolled batch jobs are the most common source of provider 429 errors in production. A RateLimitedBatcher lets you tune throughput against cost without changing caller code. The retry/fatal distinction maps directly to real provider error categories: 429 and 500 are retryable; 400 and 401 are fatal.

Constraints

Synchronous (no asyncio) — this is the version for scripts and offline batch jobs.
Use time.sleep for rate limiting and backoff. Do not mock time in your solution.
The fn callable simulates the provider — in tests, pass a function that raises on specific inputs.

Python Refresh / hard / Step 16 of 16

Practice stage

Core runtime habits

Hint

Normalize the boundary first, keep return shapes stable, and make the middle of the function boring on purpose.

Next drill

This is the end of the current mini-sequence.

Success criteria

- Returns one stable shape across branches
- Makes failure handling obvious
- Keeps the core logic readable in under a minute

Review checklist

- Did I make the input and output shapes explicit?
- Would a teammate know where validation happens?
- Did I avoid silent mutation or vague branching?

from __future__ import annotations
import time
from dataclasses import dataclass, field
from typing import Callable

class RetryableError(Exception):
    pass

class FatalError(Exception):
    pass

@dataclass
class BatchResult:
    successes: list[dict] = field(default_factory=list)
    failures: list[dict] = field(default_factory=list)
    retry_count: int = 0

class RateLimitedBatcher:
    def __init__(self, rpm_limit: int, max_retries: int = 3):
        self.rpm_limit = rpm_limit
        self.max_retries = max_retries
        self._call_times: list[float] = []

def _enforce_rate_limit(self) -> None:
        # TODO: sleep if needed to stay within rpm_limit calls per 60s
        raise NotImplementedError

def _call_with_retry(self, item: dict, fn: Callable) -> dict:
        # TODO: call fn(item), retry RetryableError up to max_retries, raise FatalError immediately
        raise NotImplementedError

def process(self, items: list[dict], fn: Callable[[dict], dict]) -> BatchResult:
        # TODO: process all items, enforce rate limit, collect successes and failures
        raise NotImplementedError

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.