Why this matters

Most implementations paste raw tool output into context and move on. This leaves significant quality on the table. Raw tool results are often poorly formatted for LLM comprehension — a database query returns raw JSON, a web search returns HTML. When these outputs land verbatim, the model works harder to extract signal, and large noisy results crowd out more important context.

Core concepts

How tool results enter the context

user:      "What's the weather in London?"
assistant: [tool_call: get_weather(location="London")]
tool:      {"temperature": 14, "condition": "cloudy", "humidity": 78}
assistant: "It's 14 degrees Celsius and cloudy in London right now."

Formatting principles

Be explicit. {"temp": 14} is less useful than Current temperature: 14 degrees Celsius.

Match format to query type. Tabular data belongs in a table, not a JSON blob.

Strip noise. HTTP headers, pagination metadata, and stack traces are rarely useful.

Handling large tool outputs

Three approaches when a tool returns more than the budget allows:

1. Extraction — pull only fields relevant to the query
2. Truncation — keep the first N tokens with a "[truncated]" marker
3. Summarization — use a fast model to compress

async def summarize_tool_result(result_text: str, query: str, client) -> str:
    response = await client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=200,
        messages=[{"role": "user", "content": (
            f"Tool result for query "{query}":
{result_text[:4000]}

"
            "Summarize in 50 words or fewer, focusing on information relevant to the query."
        )}],
    )
    return response.content[0].text

Code: ToolResultFormatter

import json
from dataclasses import dataclass
from typing import Any, Callable


@dataclass
class FormattedToolResult:
    tool_name: str
    formatted_content: str
    original_token_estimate: int
    formatted_token_estimate: int

    @property
    def compression_ratio(self) -> float:
        if self.original_token_estimate == 0:
            return 1.0
        return self.formatted_token_estimate / self.original_token_estimate


class ToolResultFormatter:
    def __init__(
        self,
        max_tokens_per_result: int = 1000,
        count_fn: Callable[[str], int] = estimate_tokens,
        summarize_fn=None,
    ):
        self.max_tokens = max_tokens_per_result
        self.count_fn = count_fn
        self.summarize_fn = summarize_fn

    def format_json_result(self, data: Any, tool_name: str) -> str:
        if isinstance(data, dict):
            lines = [f"**{tool_name} result:**"]
            for key, value in data.items():
                if key.startswith("_") or key in ("metadata", "headers", "debug"):
                    continue
                lines.append(f"- {key}: {value}")
            return "
".join(lines)
        elif isinstance(data, list):
            preview = data[:20]
            suffix = f"
[... {len(data) - 20} more items omitted]" if len(data) > 20 else ""
            lines = [f"**{tool_name} result ({len(data)} items):**"]
            lines.extend(f"- {json.dumps(item)}" for item in preview)
            return "
".join(lines) + suffix
        return f"**{tool_name} result:** {data}"

    async def process(self, tool_name: str, raw_result: Any, query: str = "") -> FormattedToolResult:
        if isinstance(raw_result, (dict, list)):
            original_text = json.dumps(raw_result, indent=2)
            formatted = self.format_json_result(raw_result, tool_name)
        else:
            original_text = str(raw_result)
            formatted = f"**{tool_name} result:**
{original_text}"

        original_tokens = self.count_fn(original_text)
        formatted_tokens = self.count_fn(formatted)

        if formatted_tokens > self.max_tokens and self.summarize_fn and query:
            summary = await self.summarize_fn(formatted, query)
            formatted = f"**{tool_name} result (summarized):**
{summary}"
            formatted_tokens = self.count_fn(formatted)
        elif formatted_tokens > self.max_tokens:
            formatted = formatted[: self.max_tokens * 4] + "
[truncated]"
            formatted_tokens = self.max_tokens

        return FormattedToolResult(
            tool_name=tool_name,
            formatted_content=formatted,
            original_token_estimate=original_tokens,
            formatted_token_estimate=formatted_tokens,
        )

    async def process_multiple(
        self, tool_results: list[tuple[str, Any]], query: str = ""
    ) -> list[FormattedToolResult]:
        import asyncio
        return await asyncio.gather(
            *[self.process(name, result, query) for name, result in tool_results]
        )

Common mistakes

Injecting raw API responses. Always extract the payload before injecting.

No token awareness. Every tool result should go through a formatter that knows the budget.

Losing information silently. Log the compression ratio and what was dropped.

Treating all JSON the same. Write format logic specific to each tool's output shape.

Try it yourself

1. Implement a format_table_result method that formats database rows as a compact markdown table.
2. Write a test that calls process() with a JSON result 3x over budget and verifies the output is within budget.
3. Call a real API and format its response verbatim vs. with ToolResultFormatter. Compare token counts.