Your personal operating system for career transition.

Build a Cost Monitoring Middleware

Without per-request cost tracking, you discover cost problems on the monthly invoice. A cost monitoring middleware intercepts every LLM call, records token usage, computes the dollar cost, and raises an alert before a budget is exceeded.

What to build

CostMonitor(llm_fn, budget_usd: float, model_prices: dict):

• model_prices format: {"gpt-4o": {"input": 0.0025, "output": 0.01}} (per 1K tokens)
• async call(model, input_tokens, output_tokens) -> dict — calls llm_fn, returns result with _cost_usd
• Raise BudgetExceededError BEFORE making the call if it would exceed the budget
• get_report() -> dict — total_usd, budget_usd, budget_remaining_usd, by_model

Constraints

• llm_fn is async: (model, input_tokens, output_tokens) -> dict
• Standard library only
• Unknown models raise ValueError