Planning and reasoning loops

Why this matters

A single LLM call can answer a question, but it cannot reliably complete a multi-step task. If you ask a model to "find the cheapest flight from NYC to London next Tuesday and book it," a single call will hallucinate an answer. An agent with a reasoning loop will break that into steps: search flights, compare prices, confirm details, then act. The reasoning loop is what separates a chatbot from an agent.

In production AI engineering, you will implement these loops constantly. Whether you are building a research assistant, a code generation pipeline, or an internal workflow tool, the pattern is always the same: think, act, observe, repeat. Getting this loop right determines whether your agent completes tasks or spirals into expensive, useless iterations.

Core concepts

ReAct (Reason + Act). The most widely used agent loop. On each iteration the model produces a thought (reasoning about what to do next), an action (a tool call), and then receives an observation (the tool result). The loop continues until the model decides it has enough information to produce a final answer.

Plan-then-execute. Instead of interleaving reasoning and action, the model first generates a complete plan (a list of steps), then executes each step sequentially. This works well when the task structure is predictable and you want to show the user a plan before executing it.

Reflection. After producing an output, the model critiques its own work and decides whether to revise. This is useful for writing, code generation, and any task where quality improves with self-review.

Stop conditions matter more than the loop itself. Every agent loop needs explicit termination criteria: a maximum number of iterations, a token budget, or a confidence signal. Without stop conditions, agents will loop forever, burning money and producing garbage.

Working example

Here is a working ReAct loop that decomposes a multi-step research question using tools.

import json
from openai import OpenAI

client = OpenAI()

# Define available tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "web_search",
            "description": "Search the web for current information on a topic.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "Search query"}
                },
                "required": ["query"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Evaluate a mathematical expression.",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {"type": "string", "description": "Math expression to evaluate"}
                },
                "required": ["expression"],
            },
        },
    },
]

def execute_tool(name: str, args: dict) -> str:
    """Execute a tool and return the result as a string."""
    if name == "web_search":
        # In production, call a real search API
        return json.dumps({"results": [f"Result for: {args['query']}"]})
    elif name == "calculate":
        try:
            result = eval(args["expression"])  # Use a safe math parser in production
            return json.dumps({"result": result})
        except Exception as e:
            return json.dumps({"error": str(e)})
    return json.dumps({"error": f"Unknown tool: {name}"})

def react_loop(question: str, max_iterations: int = 6) -> str:
    """Run a ReAct loop to answer a multi-step question."""
    messages = [
        {
            "role": "system",
            "content": (
                "You are a research assistant. Break complex questions into steps. "
                "Use tools to gather information. When you have enough information "
                "to answer confidently, provide your final answer directly."
            ),
        },
        {"role": "user", "content": question},
    ]

    for i in range(max_iterations):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools,
            tool_choice="auto",
        )

        msg = response.choices[0].message
        messages.append(msg)

        # If the model produced a final answer (no tool calls), we are done
        if not msg.tool_calls:
            return msg.content

        # Execute each tool call and feed results back
        for tool_call in msg.tool_calls:
            args = json.loads(tool_call.function.arguments)
            result = execute_tool(tool_call.function.name, args)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result,
            })

        print(f"[Iteration {i + 1}] Called: {[tc.function.name for tc in msg.tool_calls]}")

    # Hit max iterations — return whatever we have
    final = client.chat.completions.create(
        model="gpt-4o",
        messages=messages + [
            {"role": "user", "content": "Please provide your best answer now with what you have."}
        ],
    )
    return final.choices[0].message.content

# Usage
answer = react_loop(
    "What is the population of France and Germany combined, "
    "and what percentage of the EU total does that represent?"
)
print(answer)

The key structural choices: the loop has a hard iteration cap, tool results go back as tool role messages, and when the model stops calling tools the loop exits. In production you would add token counting, cost tracking, and logging at each iteration.

Common mistakes

1. No iteration limit. The agent gets confused and loops forever. Always set max_iterations and handle the case where you hit it.
2. Letting the model plan in its head. If you skip the system prompt guidance to break things into steps, the model will try to answer in one shot and hallucinate.
3. Not logging intermediate steps. When the agent gives a wrong final answer, you need the trace of thoughts and tool calls to debug it. Log every iteration.
4. Overly complex plans. Plan-then-execute works poorly when the model generates 15 steps. Encourage short plans (3-5 steps) and allow re-planning.
5. Ignoring cost. Each iteration is an LLM call. A 10-iteration loop on GPT-4o with long context can cost dollars per query. Track and budget.

Try it yourself

Build a ReAct loop that answers a question requiring two different tools — for example, "What is the current stock price of AAPL multiplied by the number of employees at Apple?" Make the agent search for each fact separately and then combine them with a calculation tool. Watch the trace to see how the model decomposes the problem.