Tool design for LLM agents

Why this matters

Every agentic system lives or dies by the quality of its tools. When a model decides to call a function, it relies entirely on the schema you gave it — the name, the description, the parameter types, and what comes back. Sloppy tool definitions create cascading failures: the model hallucinates parameters, misinterprets results, or retries endlessly. If you are building agent systems professionally, tool design is the single highest-leverage skill you can sharpen.

In production, a well-designed tool means fewer wasted tokens, fewer retries, lower latency, and more predictable behavior. A poorly designed tool means debugging sessions where you stare at traces wondering why the model called search_database with a date string where an integer was expected.

Core concepts

JSON Schema for function definitions. Every major LLM provider (OpenAI, Anthropic, Google) uses JSON Schema to describe tools. The schema tells the model what parameters exist, their types, which are required, and what the function does. The model generates a JSON object matching this schema, and your code executes the function.

Key principles for good tool schemas:

• Descriptive names and descriptions. The model reads these. search_orders is better than query_db. Add a description that explains when to use it, not just what it does.
• Minimal required parameters. Every required field the model must fill is a chance for it to hallucinate. Keep the interface tight.
• Enum constraints. If a parameter has known valid values, use an enum. The model will pick from the list instead of inventing values.
• No nested complexity. Flat parameter objects work best. Deeply nested schemas confuse models and increase error rates.

Return contracts. Your tool should always return a structured response the model can parse. Never return raw HTML, giant blobs of text, or unstructured error messages. Define a return shape and stick to it.

Error envelopes. When a tool fails, the model needs to understand what happened so it can decide whether to retry, try a different approach, or report the failure. Wrap errors in a consistent envelope with a status, error type, and human-readable message.

Working example

Here is a complete tool definition with validation, retry logic, and structured error handling — the kind of function you would register with an agent framework.

import json
import time
import httpx
from pydantic import BaseModel, Field
from enum import Enum
from typing import Optional

# --- Return contract ---
class ToolStatus(str, Enum):
    success = "success"
    error = "error"

class ToolResult(BaseModel):
    status: ToolStatus
    data: Optional[dict] = None
    error_type: Optional[str] = None
    error_message: Optional[str] = None

# --- The tool function ---
def lookup_order(order_id: str, include_items: bool = False) -> dict:
    """Look up an order by its ID. Use this when the user asks about
    a specific order status, shipment, or order details.

    Args:
        order_id: The order ID (format: ORD-XXXXX)
        include_items: Whether to include line items in the response
    """
    # Input validation
    if not order_id.startswith("ORD-") or len(order_id) != 9:
        return ToolResult(
            status=ToolStatus.error,
            error_type="validation_error",
            error_message=f"Invalid order ID format: {order_id}. Expected ORD-XXXXX.",
        ).model_dump()

    # Retry logic for transient failures
    max_retries = 2
    for attempt in range(max_retries + 1):
        try:
            resp = httpx.get(
                f"https://api.internal.co/orders/{order_id}",
                params={"items": include_items},
                timeout=5.0,
            )
            resp.raise_for_status()
            return ToolResult(
                status=ToolStatus.success,
                data=resp.json(),
            ).model_dump()
        except httpx.TimeoutException:
            if attempt < max_retries:
                time.sleep(1.5 ** attempt)
                continue
            return ToolResult(
                status=ToolStatus.error,
                error_type="timeout",
                error_message="Order service timed out after retries.",
            ).model_dump()
        except httpx.HTTPStatusError as e:
            return ToolResult(
                status=ToolStatus.error,
                error_type=f"http_{e.response.status_code}",
                error_message=f"Order service returned {e.response.status_code}.",
            ).model_dump()

# --- JSON Schema for the LLM provider ---
TOOL_SCHEMA = {
    "type": "function",
    "function": {
        "name": "lookup_order",
        "description": (
            "Look up an order by its ID. Use when the user asks about "
            "a specific order's status, shipment tracking, or line items."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order ID in the format ORD-XXXXX",
                    "pattern": "^ORD-[A-Z0-9]{5}$",
                },
                "include_items": {
                    "type": "boolean",
                    "description": "Include line items in the response",
                    "default": False,
                },
            },
            "required": ["order_id"],
        },
    },
}

Notice the pattern: validate inputs before doing any work, retry on transient failures only, and always return the same ToolResult shape. The model sees status: error and can decide what to do next — it never has to parse a Python traceback.

Common mistakes

1. Returning raw exceptions. If your tool returns "ConnectionError: [Errno 111] Connection refused", the model will try to interpret that string creatively. Use structured error envelopes.
2. Giant return payloads. Returning 50KB of JSON eats your context window. Trim results to what the model actually needs to answer the user's question.
3. Vague descriptions. "Searches the database" tells the model nothing about when to use this tool. Be specific about the use case.
4. Missing validation. If the model can pass a string where you expect an integer, it will — eventually. Validate early and return a clear error.
5. No retry for transient errors. Network calls fail. A single retry with backoff prevents the agent from giving up on a momentary blip.

Try it yourself

Take an API you already use (weather, GitHub, a database) and wrap it as an agent tool. Define the JSON Schema, add input validation, implement retry logic, and return structured results. Then feed the schema to an LLM and see if it calls the tool correctly on the first try. If it does not, look at what confused it — that is where your schema needs work.