Your personal operating system for career transition.

Validate LLM Structured Output with Pydantic

LLMs frequently return structured data that does not perfectly match what you asked for. A model asked to extract entities might wrap its JSON in a markdown code fence. It might omit required fields. It might return a string where you expect a list. Your application code should never let these failures propagate silently.

What to build

Implement a parse_llm_output function that robustly parses and validates LLM-generated structured data:

1. Strip markdown code fences — handle responses wrapped in ```json or ``` blocks.
2. Extract the JSON object or array — find the outermost {...} or [...] even when surrounded by prose.
3. Validate against a Pydantic model — use model_class.model_validate() to check the parsed JSON.
4. Return a typed result — return a ParseResult dataclass with: success: bool, data: BaseModel | None, error: str | None, raw: str.

Also implement ExtractedFacts, a Pydantic model for the test case:

• subject: str — required, minimum length 1
• claims: list[str] — required, minimum 1 item
• confidence: float — required, between 0.0 and 1.0

Why this matters

LLM structured outputs fail in the same small set of predictable ways. Building one robust parser you can reuse across many prompt tasks is far better than adding ad-hoc cleanup logic everywhere. Explicit ParseResult return types give callers the information they need to decide how to handle failures — log and skip, retry, or escalate to a human reviewer.

Constraints

• Do not use eval() or ast.literal_eval() to parse JSON — use json.loads().
• Handle the case where JSON extraction succeeds but Pydantic validation fails.
• All fields of ParseResult should be populated even on failure.