Cut agent-loop turns and defer batchable stages

Learning objective

Reduce agent-loop cost with fewer turns, parallel and programmatic tool calls, and batch processing for latency-tolerant work.

Core idea

Independent work can run together, while dependent work needs its inputs first. Batch processing suits work whose consumer accepts deferred completion; those scheduling principles still apply when you optimize the internal agent loop.

The professional decision is where a model genuinely needs to inspect a result before the workflow can proceed. A model turn that only forwards data or repeats an established calculation adds generation, replayed input, and waiting. Compare three placements of control: Claude selecting several direct calls, Claude writing code that coordinates tools, and an application scheduling self-contained work for later processing.

How it works

Remove turns that contribute no decision

Start with a trace of one completed task. Mark every request, tool call, result, and final acceptance check. For each model response, identify the decision it makes. If a step always fetches the same independent records, does deterministic filtering, and then asks the model to summarize the result, consider moving that routine orchestration into a task-shaped tool or programmatic execution.

Anthropic’s tool-design guidance recommends considering the agent’s task rather than wrapping every backend operation as a separate tool. For example, a custom tool can expose a search with filters and return relevant records in one operation instead of forcing the model to alternate between listing, inspecting, and filtering each record. Its name, description, and input_schema must make that larger operation understandable. Keep authorization in the handler; a convenient compound operation must not silently acquire broader access.

Measure whether fewer turns actually reduce cost per accepted result. A compound tool that always retrieves every possible record can replace several calls while increasing backend work and response volume. Conversely, one extra targeted call can be cheaper than a large omnibus result. Retain intermediate decisions that require interpretation or permission rather than collapsing them for a lower turn count.

Choose direct parallel calls when the model needs the results

Use the direct parallel tool_use exchange from 3.3.2 for independent lookups; here the cost benefit comes from avoiding separate model turns and replaying the same context between them.

Do not set tool_choice.disable_parallel_tool_use: true when the intended design requires multiple calls in one response. That switch limits calls, not your application’s concurrency scheduler. Parallel calls still spend their tool-service cost, and their returned data still enters model context. Concurrency alone is not a promise of fewer billed tokens.

Use the direct path when Claude must inspect the returned records individually before deciding what to do. Execute writes or shared-state operations in the required order. For calls you intentionally skip, return is_error: true with the reason instead of leaving an unmatched call in history.

Use programmatic calls for routine data processing

Programmatic tool calling lets Claude write code that invokes tools inside the code execution container, processes their results, and returns only the final code output to model context. Intermediate programmatic tool results do not count toward model input/output token usage. This removes model sampling between routine calls and avoids filling context with records used only for filtering or aggregation.

On the Claude API, include a code execution tool with type: "code_execution_20260120" and name: "code_execution", then mark an eligible custom tool with allowed_callers: ["code_execution_20260120"]. The feature is generally available, requires that tool version or later, and supports specified models; Claude Haiku 4.5 does not support programmatic calling even though it accepts those code execution versions. Do not assume that code execution support alone proves compatibility. The documentation lists support on the Claude API, Claude Platform on AWS, and Microsoft Foundry with a Hosted on Anthropic deployment; it does not list Amazon Bedrock or Google Cloud. Programmatic tool calling is not eligible for zero data retention (ZDR), which rules it out when the delivery requires ZDR.

Your application still executes its custom tools. A returned tool_use.caller.type identifies a code-execution invocation. When returning a pending programmatic result, pass the response’s container ID and send a user message containing only tool_result blocks, with text-only result content. Omitting that container is rejected. Pending calls time out after about four minutes inside the code, so an interactive programmatic continuation is unsuitable for a worker that might wait hours to return data.

Intermediate programmatic tool results therefore pass between the code execution container and your application, and only the final code output returns to model context.

Treat allowed_callers as presentation guidance, not authorization: the documentation says it is not a hard API-level block on direct invocation. Your handler must enforce access regardless of caller. Validate external results as data rather than executing supplied strings. Tools provided through the MCP connector cannot be called programmatically; the computer-use and browser-use toolsets are also direct-only. Programmatic calling also does not support tools marked strict: true or requests with tool_choice.disable_parallel_tool_use: true.

Programmatic execution introduces code-execution pricing and another failure surface. Anthropic reports that it can increase cost on sequential single-call workloads. Choose it for repeated calls and substantial intermediate processing, then measure the generated code, final output, execution charges, and quality together. A documented benchmark saving is not your workload’s forecast.

Defer eligible stages rather than the entire loop

Message Batches charges 50% of standard API prices, with a 24-hour expiration boundary rather than guaranteed successful completion. Apply that economic option to self-contained latency-tolerant stages, not automatically to every stage in an agent workflow.

Submit the selected report stage through POST /v1/messages/batches, retaining its custom_id in the task record. Evaluate the stage boundary as part of the optimization: data collection must finish before submission, and any dependent synthesis follows the returned results. Include staging storage, quality reruns, and recovery before the publication cutoff in the comparison. The report owner must also decide whether data collected at submission remains fresh enough when the report is consumed. A discounted stage that triggers a complete refresh can erase its apparent saving.

Key distinctions and trade-offs

Choose by what must reach the next decision:

These choices can coexist. Parallel execution changes the schedule, programmatic execution changes where intermediate data is processed, and batching changes delivery timing and price. Judge the combined workflow against 4.5.1, including tasks that require recovery.

Architecture / decision scenario

A technology firm’s procurement assistant must answer an analyst within twelve seconds, while a portfolio report is read the next afternoon. On 500 fixed cases, direct serial lookups take eighteen seconds p95. Parallel direct calls reach ten seconds but return large records. Programmatic filtering reaches eight seconds and lowers measured task cost by 22%, while retaining the same 97% acceptance. These are illustrative results.

The architect chooses programmatic calling for repeated license lookups, preserves direct calls where Claude must examine an exception, and keeps purchase approval sequential after the findings. Moving the whole workflow into batches would violate the analyst deadline. Instead, the reporting owner batches the self-contained portfolio summaries after data collection, with explicit recovery for missing results before publication.

The integration owner handles container continuations and tool failures; the procurement owner reviews missed exceptions. If the workload shifts toward single-call requests, the team rechecks whether programmatic overhead still earns its place.

Common exam traps

• Equating fewer turns with fewer tool calls or lower backend cost.
• Expecting parallel calls to reduce the tokens that tool results add to context.
• Treating allowed_callers as an authorization boundary.
• Returning a pending programmatic result without its container ID.
• Selecting programmatic execution for every single-call workflow.
• Deferring a stage whose consumer needs fresher data than its collected inputs provide.

What to remember

Spend model turns on decisions that require them. Combine independent direct calls, use programmatic processing when intermediate data need not enter context, and batch only eligible deferred stages. Keep dependencies, execution costs, and accepted quality in the comparison.

Knowledge check

References

• Anthropic, Writing effective tools for agents — with agents
• Anthropic, Parallel tool use
• Anthropic, Programmatic tool calling
• Anthropic, Batch processing