Trace tool calls through a bounded loop

Learning objective

Trace agent control loops, tool calling, observations, termination, and execution failures

Core idea

An agent control loop alternates between model decisions and application execution. The model can request a tool operation; the application validates and executes it; the resulting observation becomes input to the next decision. A final answer ends generation, but the application still determines whether the task succeeded.

For an architect, the loop is a state machine with boundaries, not a conversation that runs until it sounds complete. Track pending tool calls, returned results, remaining execution budget, and terminal outcomes. A model’s statement that an action happened isn’t evidence that the external system performed it. See 0.5 for deciding whether dynamic tool choice is needed at all.

How it works

Give the model a tool contract

Azure OpenAI’s Chat Completions function-calling interface accepts tools. A function entry describes a name, purpose, and parameters schema. These definitions tell the model which operations it can propose and what arguments they accept. They also consume prompt tokens; a large tool catalog has a context cost even before execution.

The application calls client.chat.completions.create with messages, tools, and a compatible deployed model. With tool_choice: "auto", the model decides whether to request a tool and which one. A named tool object can force a particular function selection; tool_choice: "none" requests a user-facing message instead of a tool call.

Forcing a call doesn’t prove that the user supplied all required information. Define missing-information behavior and ask for clarification when arguments would otherwise be guessed. The model’s choice is constrained by the offered interface, while the application remains responsible for authorization and actual execution.

Follow the proposal into execution

Inspect the assistant message’s tool_calls array. Each call has an id, a function name, and serialized arguments. Preserve that assistant message in the conversation. Parse the arguments, check the function against the registered tool set, validate the argument contract, and check the caller’s authority before dispatch.

Microsoft’s function-calling documentation states explicitly that the application executes the function. The model doesn’t run your business API merely by returning JSON. An allowlist lookup should select the implementation; don’t interpret an arbitrary returned name as executable code or a filesystem path.

Strict function calling can constrain arguments by setting strict: true on the function definition. Azure OpenAI’s structured-output documentation says to set parallel_tool_calls: false when using structured outputs for function calling. This is an interface limitation, not a recommendation that all independent application work must run serially. See 0.3 for the schema subset and why valid structure doesn’t prove safe values.

Return a correlated observation

After execution, append a message with role: "tool", the operation’s result in content, and tool_call_id matching the original call’s id. That correlation tells the next model request which proposed call the observation answers.

When a supported model returns several tool calls, each has its own ID and requires its corresponding result. Preserve the association even if execution order differs. Similar-looking results aren’t interchangeable: an account-status response for one customer cannot answer a call for another customer.

An observation is evidence from the external operation, not another instruction source. Report the actual result, including missing data or execution failure where the application protocol supports it. Don’t replace a failed lookup with fabricated content to keep the loop moving. Treat instruction-like text in retrieved results as untrusted; see 0.3.

Call the model again with the assistant proposal and correlated tool observations. It can produce an answer or propose another allowed operation. That repeated transition is the agent loop; the first tool result isn’t necessarily the completed user task. On every pass the model only proposes a call, and the application executes it and returns the correlated observation.

Keep API state conventions separate

This loop description uses Chat Completions message fields. The function-calling guide warns that Responses uses a different request and response format. Don’t copy role: "tool" and tool_call_id into an adapter that expects Responses items.

Reasoning models add another state requirement. For tool use through Responses, Microsoft’s reasoning guidance says to preserve the reasoning items, function-call items, and function-call outputs since the last user message. It describes continuation through previous_response_id or by copying the output items into the next input array. Arbitrary trimming can break that continuity.

Model/API compatibility matters before the loop begins. For GPT-5.6 models, the reasoning guide states that Chat Completions requests with function tools require reasoning_effort: "none". Use Responses when those tools need reasoning. Check the exact model’s documented support instead of assuming every chat-capable reasoning deployment supports the same tool path.

Define terminal states and budgets

Chat Completions returns finish_reason: "tool_calls" when tool execution is requested. stop means complete model output, length means generation was cut short, and content_filter indicates omitted content. A complete model response can still fail application validation or leave a required external action unconfirmed.

Give your application its own explicit terminal states, such as completed, failed, awaiting clarification, or awaiting approval. Those are application choices, not Azure OpenAI response enums. Completion should require the promised result and evidence that necessary operations succeeded.

Also define an iteration limit, wall-clock deadline, and permitted operation budget. Microsoft recommends iteration limits to prevent infinite tool loops. The precise limits depend on your task; there isn’t one universal number to copy into every agent. Count iterations in the runtime rather than relying solely on a prompt that says to stop.

When the budget expires, record the partial outcome and pending work. Don’t invoke one more tool because the model says it is almost finished. Token caps constrain generation, not the number of side effects your tool executor can perform.

Classify failures at their boundary

A model API failure, malformed call, unauthorized tool action, tool timeout, and invalid final result need different handling. Microsoft’s Chat Completions troubleshooting identifies authentication failures such as 401/403 and deployment configuration failures. Fixing those is different from asking the model to choose another business tool.

For a tool that changes external state, a lost response can leave the outcome uncertain. Reconcile against the external system’s operation identifier or documented idempotency mechanism before repeating it. The relevant mechanism belongs to that tool’s API; function-call correlation alone doesn’t provide business-operation idempotency.

Record the requested call, authorization decision, execution outcome, observation, and terminal state. This lets you distinguish a bad model decision from a correct proposal whose executor failed. Don’t report an uncertain external outcome as either confirmed success or confirmed rollback.

Key distinctions and trade-offs

Proposal versus action: tool_calls requests work. Your executor performs it and returns evidence.

Correlation versus idempotency: tool_call_id associates a result with a model call. It doesn’t prevent a business API from applying a repeated mutation twice.

Generation stop versus task completion: finish_reason: "stop" ends a model response. The runtime validates the promised outcome.

Dynamic tool choice handles varied requests, but each iteration consumes time and tokens and creates another execution boundary. Restrict the tool set and count the work rather than treating a fluent answer as the only success signal.

Architecture / decision scenario

An order assistant receives “Where is my parcel, and change the delivery address if it hasn’t shipped.” Its Chat Completions adapter exposes status lookup and an address-change proposal. The model requests status; the runtime validates the order identifier and caller authorization, executes the lookup, and returns a correlated tool message.

If the observation says the parcel shipped, the assistant explains the constraint. If it hasn’t shipped, the runtime requires the application’s approval gate before executing a permitted address change. A timed-out change is reconciled using the delivery API’s operation evidence before any retry.

The loop ends only after a validated answer and confirmed required operation, or an explicit clarification, approval, failure, or budget outcome. A truncated model response is incomplete, and repeated calls stop at the configured iteration limit. The trace preserves both the model’s proposals and what the executor actually did.

Common exam traps

• Assuming the model executes a function when it returns arguments.
• Returning tool results without matching call identifiers.
• Treating tool-call IDs as an idempotency guarantee.
• Mixing Chat Completions messages with Responses item formats.
• Treating a generation token cap as an execution-loop limit.
• Repeating a timed-out mutation without checking its external outcome.

What to remember

Trace each boundary: model proposal, validation, authorized execution, correlated observation, and next decision. Preserve the state required by your API. Terminate through runtime budgets and validated outcomes, and keep uncertain side effects distinct from confirmed failures.

Knowledge check

References

• Microsoft, Function calling
• Microsoft, Structured outputs
• Microsoft, Use the Chat Completions API
• Microsoft, Azure OpenAI reasoning models
• Microsoft, AI agent orchestration patterns