AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Parse structured outputs with error recovery

Parse Structured Outputs with Error Recovery

LLMs are asked to produce JSON constantly — tool call arguments, structured answers, classification labels, extraction results. But LLM output is unreliable: it might be wrapped in markdown code fences, contain trailing commas, include explanatory text before or after the JSON, or be truncated mid-object.

What you are building

Create a parse_structured_output function and supporting utilities that:

Extract JSON from markdown — strip ```json ... ``` fences and other common wrappers.
Handle partial/truncated JSON — attempt to close unclosed braces and brackets to salvage partial responses.
Validate against a schema — check the parsed object against a provided JSON Schema and return clear validation errors.
Apply defaults — fill in missing optional fields with schema-defined defaults.
Return a structured result — include the parsed data, whether recovery was needed, and any warnings.

Why this matters

In agent systems, every tool call argument and every structured response passes through a parsing step. If that step is fragile, your agent breaks on edge cases that are actually common in practice: the model wraps JSON in markdown 30% of the time, truncation happens when hitting token limits, and extra text before JSON is routine with weaker models.

Robust parsing is not a nice-to-have — it is what separates agents that work in demos from agents that work in production.

Constraints

Do not use an LLM to fix the output — this must be deterministic.
Handle at least: markdown fences, leading/trailing text, single trailing comma, unclosed braces/brackets (up to 3 levels).
Return warnings for every recovery action taken so callers can log and monitor parse quality.

Agents / intermediate / Step 4 of 8

Practice stage

Agent architecture patterns

Hint

Focus on explicit control surfaces — function schemas, state machines, and structured outputs. Agents are most useful when the task requires dynamic tool selection or multi-step reasoning. Start with the simplest pattern (single tool call) before reaching for ReAct loops.

Next drill

Conversation memory manager with token budgeting

Success criteria

- Tools return structured, typed responses
- Agent completes the task within a bounded number of steps
- All tool calls include error handling and retries
- Memory/state management prevents unbounded context growth

Review checklist

- Tool schemas validate inputs and handle errors gracefully
- Agent loop has explicit termination conditions
- State is serializable and inspectable between steps
- Cost and token usage are tracked per invocation
- Fallback behavior exists for tool call failures

from __future__ import annotations

import json
import re
from dataclasses import dataclass, field
from typing import Any

@dataclass
class ParseResult:
    """Result of parsing structured LLM output."""
    success: bool
    data: Any = None
    recovered: bool = False
    warnings: list[str] = field(default_factory=list)
    error: str | None = None

def strip_markdown_fences(text: str) -> str:
    """Remove markdown code fences from around JSON content."""
    # TODO: handle ```json ... ```, ```... ```, and plain ``` fences
    raise NotImplementedError

def extract_json_substring(text: str) -> str:
    """Find the first JSON object or array in a string with surrounding text."""
    # TODO: find the first { or [ and its matching closer
    raise NotImplementedError

def repair_truncated_json(text: str) -> tuple[str, list[str]]:
    """Attempt to close unclosed braces and brackets. Return repaired text and warnings."""
    # TODO: count open/close braces and brackets, append closers
    # TODO: handle trailing commas before closers
    raise NotImplementedError

def parse_structured_output(
    raw: str,
    schema: dict[str, Any] | None = None,
) -> ParseResult:
    """
    Parse structured JSON from raw LLM output with recovery.

Steps:
    1. Strip markdown fences
    2. Extract JSON substring
    3. Try parsing; if it fails, attempt repair
    4. Validate against schema if provided
    5. Apply defaults from schema
    """
    # TODO: implement the pipeline above
    raise NotImplementedError

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.