AI Engineer Portal

Guardrails and structured outputs

Why this matters

An LLM that returns free text is useful for chat. An LLM feature that powers a form, a workflow, or a database write needs structured, validated output. The gap between "the model usually returns JSON" and "my application can reliably parse and act on model output" is where most production LLM bugs live.

Guardrails work in both directions. Input guardrails protect your system from malicious or malformed user requests. Output guardrails ensure the model's response is safe, structured, and complete before it flows into downstream code.

Core concepts

Three approaches to structured output:

JSON mode: Tell the model to respond only with valid JSON. Every major provider supports this. The model is unconstrained on which keys it uses, so you still need to validate the shape.
Function calling / tool use: Define the expected output shape as a JSON Schema and ask the model to call a function with that schema as its arguments. The provider enforces that the output matches the schema. This is the most reliable approach.
Prompted JSON: Ask for JSON in the system prompt without using any provider feature. The least reliable approach — models sometimes add explanation text before or after the JSON block.

For production features, use function calling / tool use. It gives you provider-enforced structure and explicit error messages when the model cannot comply.

Pydantic validation. Even with function calling, the model can produce unexpected field values. Run every structured output through Pydantic before acting on it:

from pydantic import BaseModel, Field, field_validator
from typing import Optional
from enum import Enum

class Sentiment(str, Enum):
    positive = "positive"
    neutral = "neutral"
    negative = "negative"

class SupportTicketClassification(BaseModel):
    category: str = Field(..., description="Issue category")
    sentiment: Sentiment
    priority: int = Field(..., ge=1, le=5, description="Priority 1-5, 5 being highest")
    requires_human_review: bool
    summary: str = Field(..., max_length=200)

    @field_validator("category")
    @classmethod
    def normalize_category(cls, v: str) -> str:
        return v.lower().strip()

Input guardrails. User input can contain:

Prompt injection attempts ("Ignore previous instructions and...")
Content that violates your terms of service
Inputs that will produce poor or dangerous responses

A basic injection detection layer:

import re

INJECTION_PATTERNS = [
    r"ignore (all |previous |above )?instructions",
    r"disregard (your |the )?(previous |above |system )?prompt",
    r"you are now",
    r"act as (a |an )?(?!assistant)",
    r"DAN mode",
    r"jailbreak",
]

def check_prompt_injection(user_input: str) -> tuple[bool, str]:
    """Returns (is_safe, reason)."""
    lower = user_input.lower()
    for pattern in INJECTION_PATTERNS:
        if re.search(pattern, lower):
            return False, f"Matched injection pattern: {pattern}"
    if len(user_input) > 10000:
        return False, "Input exceeds maximum length"
    return True, "ok"

Output guardrails. After the model responds:

Validate that required fields are present
Check for hallucinated values (e.g., dates in the future, impossible numbers)
Enforce format constraints (phone numbers, emails, codes)
Detect refusals or uncertainty signals

Working example

A structured output parser with Pydantic that handles malformed responses gracefully:

import json
import re
import anthropic
from pydantic import BaseModel, ValidationError
from typing import Optional
import logging

logger = logging.getLogger(__name__)
client = anthropic.Anthropic()

class ProductReview(BaseModel):
    sentiment: str  # "positive", "neutral", "negative"
    score: int       # 1-5
    key_themes: list[str]
    would_recommend: bool
    response_text: str

def extract_json_from_text(text: str) -> Optional[dict]:
    """Extract JSON from a text that may contain surrounding prose."""
    # Try direct parse first
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        pass

    # Look for JSON blocks in markdown code fences
    fence_match = re.search(r"```(?:json)?\s*({.*?})\s*```", text, re.DOTALL)
    if fence_match:
        try:
            return json.loads(fence_match.group(1))
        except json.JSONDecodeError:
            pass

    # Look for bare JSON objects
    obj_match = re.search(r"({[^{}]*})", text, re.DOTALL)
    if obj_match:
        try:
            return json.loads(obj_match.group(1))
        except json.JSONDecodeError:
            pass

    return None

def parse_review_analysis(raw_text: str, fallback_text: str) -> ProductReview:
    """Parse model output into a validated ProductReview, with fallback."""
    # Try to extract and validate
    raw_dict = extract_json_from_text(raw_text)
    if raw_dict is not None:
        try:
            return ProductReview(**raw_dict, response_text=raw_text)
        except (ValidationError, TypeError) as e:
            logger.warning(f"Pydantic validation failed: {e}")

    # Fallback: return a safe default
    logger.warning("Could not parse structured output, using fallback")
    return ProductReview(
        sentiment="neutral",
        score=3,
        key_themes=[],
        would_recommend=False,
        response_text=fallback_text,
    )

def analyze_review(review_text: str) -> ProductReview:
    """Analyze a product review using tool use for structured output."""
    # Input guardrail
    is_safe, reason = check_prompt_injection(review_text)
    if not is_safe:
        logger.warning(f"Input guardrail blocked request: {reason}")
        return ProductReview(
            sentiment="neutral", score=3,
            key_themes=[], would_recommend=False,
            response_text="Input blocked by safety filter.",
        )

    response = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=512,
        tools=[{
            "name": "analyze_review",
            "description": "Analyze a product review and return structured sentiment data.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "sentiment": {
                        "type": "string",
                        "enum": ["positive", "neutral", "negative"],
                    },
                    "score": {"type": "integer", "minimum": 1, "maximum": 5},
                    "key_themes": {
                        "type": "array",
                        "items": {"type": "string"},
                        "maxItems": 5,
                    },
                    "would_recommend": {"type": "boolean"},
                },
                "required": ["sentiment", "score", "key_themes", "would_recommend"],
            },
        }],
        tool_choice={"type": "tool", "name": "analyze_review"},
        messages=[{"role": "user", "content": f"Analyze this review:\n\n{review_text}"}],
    )

    # Extract tool use result
    for block in response.content:
        if block.type == "tool_use":
            try:
                return ProductReview(**block.input, response_text=review_text)
            except (ValidationError, TypeError) as e:
                logger.error(f"Tool output validation failed: {e}\nInput: {block.input}")
                return parse_review_analysis(str(block.input), review_text)

    # Model did not call the tool (unusual with tool_choice forced)
    return parse_review_analysis(response.content[0].text if response.content else "", review_text)

The critical pattern here: use tool_choice={"type": "tool", "name": "..."} to force the model to call your tool and never return free text. Then validate with Pydantic. Then have a fallback for when validation fails. Defense in depth.

Common mistakes

Trusting JSON mode without schema validation. JSON mode guarantees valid JSON — it does not guarantee that the JSON has the fields you need. Always validate with Pydantic.
No fallback on validation failure. If Pydantic throws ValidationError and your code does not catch it, a malformed model response crashes your feature. Every structured output parser needs a fallback path.
Injection in the system role via template interpolation. A template like f"You are {user.name}'s assistant" where user.name is user-controlled is an injection surface. Validate and sanitize before interpolating.
Checking for injection in the wrong place. Injection detection should happen before any prompt assembly, not after. Once malicious content is assembled into a prompt, the damage is already possible.
Overly strict Pydantic schemas. A schema that rejects any output where score is a float (e.g., 4.0 instead of 4) will fail on valid model responses. Use coerce_numbers_to_str=True or field validators to handle common type mismatches.

Try it yourself

Build an output guardrail that detects refusals. LLMs sometimes respond with "I cannot help with that" or "I don't have information about..." even when you expected structured JSON. Write a function that detects common refusal patterns in model output and either retries with a clarified prompt or returns a structured fallback. Test it by crafting prompts that deliberately trigger refusals.