AnthropicClaude Certified Architect – Professional2026-104.5 Optimize token usage, latency, and cost-performance trade-offs
Tune generated work before capping output
Completed by the knowledge check
Learning objective
Control output and reasoning tokens with concise formats, output limits, and effort settings.
Core idea
Use three controls for three different decisions: the requested format defines the answer the consumer needs, max_tokens limits total generated output, and output_config.effort influences how much work Claude spends responding. A small ceiling cannot substitute for a concise answer specification or a measured reasoning setting.
The architectural objective is lower cost per accepted output, including incomplete attempts and repairs. See 2.1.3 for choosing supported thinking controls. Here you design a production output policy that combines those controls, measures their interactions, and preserves difficult-case quality.
How it works
Specify the smallest useful answer
Start with the downstream consumer. A classifier might need a label and a short reason; an analyst might need a conclusion, supporting sources, and unresolved issues. Remove decorative narrative before removing fields that make the result actionable or reviewable. An answer that omits uncertainty can look efficient while transferring risk to the reader.
For prose, request a sentence or paragraph count in messages or standing instructions rather than relying on an exact word limit. Anthropic’s latency guidance explains that word-count limits are less effective than sentence or paragraph limits because models operate on tokens. Evaluate whether a short opening followed by detail on request suits the workflow; if most readers immediately ask for expansion, count that second turn in the cost comparison.
For machine consumers, use output_config.format with type: "json_schema" and a schema containing the required fields. On supported models this constrains response structure; it does not establish factual correctness. A verbose schema with several redundant explanation fields still asks for verbose output. Keep a reason or source field when it supports validation, and remove it only when the business result remains verifiable elsewhere.
Structured outputs are generally available on the Claude API for supported models. Platform support differs: the documentation supports specified models on the legacy Amazon Bedrock integration, but not Claude in Amazon Bedrock. Check the deployed route rather than assuming that JSON output support follows every Claude deployment. This lesson’s request fields describe the Claude API.
Use a ceiling without rewarding truncation
max_tokens is a hard ceiling on generated output, including thinking plus the final response. It is not a reservation you pay in full merely by setting it. The cap is invisible to the model, so lowering it does not make Claude economize on generation; cut-off turns are still billed. Leave enough headroom for the output specification and the reasoning configuration, then inspect actual consumption.
When a response ends with stop_reason: "max_tokens", treat the result as potentially incomplete. For structured output, the documented remedy is to retry with a higher max_tokens value to obtain the complete object. Record the first attempt and the retry in task spending. Accepting malformed or partial output to preserve a token-saving chart defeats the objective.
Structured output can also return stop_reason: "refusal"; a schema does not override that outcome. Separate refusal handling from truncation recovery. Raising the cap addresses missing generation room, not a refusal. Do not put both into an undifferentiated “retry until JSON parses” loop.
Truncation and refusal therefore take separate paths, and only truncation is retried with a higher max_tokens value.
Set a generous ceiling rather than tuning it as a savings target. Anthropic’s cost guide recommends max_tokens: 64000 for agentic work; check that the deployed model supports that output allowance and stream such large responses. Treat stop_reason: "max_tokens" as a failed attempt, not successful brevity. The service owner should obtain savings through the format and effort instructions Claude receives, then confirm that the ceiling leaves room for difficult cases without truncation.
Sweep effort against the same output contract
Use the model-supported top-level output_config.effort setting selected in 2.1.3 explicitly for comparable runs; per-message effort is a separate beta capability outside this tuning policy.
Effort affects response text, tool calls and their arguments, and thinking when active. It is a behavioral signal rather than a strict token budget. Lower effort therefore can change both spending and how thoroughly the workflow searches or verifies. It also does not reliably shorten visible responses on every model, so retain the concise output instructions.
Keep the prompt, schema, and generous cap fixed while sweeping effort. Then choose a ceiling that accommodates the winning configuration and re-evaluate. Changing effort and lowering the cap simultaneously can make a valid effort level appear inadequate when the actual cause is truncation. Conversely, a large cap does not force a model at low effort to spend it.
Key distinctions and trade-offs
The controls divide by the failure they address:
| Control | What you decide | What still needs testing |
|---|---|---|
| Concise instructions or JSON schema | Necessary visible content | Missing qualifications, factual errors, follow-up demand |
max_tokens | Maximum total generation | Truncation, complete objects, costly repairs |
output_config.effort | Depth and volume of generated work | Difficult-case reasoning and tool-use quality |
Use usage.output_tokens for total output and usage.output_tokens_details.thinking_tokens for its reasoning subset, as established in 4.5.1. A shorter visible answer with unchanged reasoning may save little. A lower reasoning total accompanied by extra tool turns might also fail to reduce task spending. Measure both rather than choosing a control from visible response length alone.
Architecture / decision scenario
An education provider produces advisor briefs with a four-second p95 target and a 98% acceptance floor. The academic-policy owner requires the recommended action, policy source, and unresolved eligibility facts. The platform architect considers a tiny output cap, a concise JSON brief, and lower effort.
On the same 800 cases, the baseline accepts 99% at 5.5 seconds p95. Cutting the cap alone reaches 3.1 seconds but produces incomplete objects on 7% of cases. A schema containing the three necessary fields, adequate output headroom, and medium effort accepts 98.8% at 3.8 seconds. low effort reaches 3.2 seconds but drops acceptance to 96%, concentrated in conflicting eligibility rules. These are illustrative measurements.
The architect chooses the concise medium configuration and rejects both faster alternatives. Before release, the academic-policy owner reviews the conflict slice; the service owner records the cap and repair rule. Monitoring distinguishes max_tokens, refusals, rubric failures, and expansion requests, so a smaller answer cannot conceal rising human follow-up costs.
Common exam traps
- Treating
max_tokensas a target answer length or a prepaid token amount. - Expecting a schema to guarantee factual correctness or prevent refusals.
- Removing source and uncertainty fields solely to shorten output.
- Assuming effort is a hard reasoning-token cap.
- Carrying the same default or effort setting across every model.
- Measuring only the visible answer while ignoring thinking and repairs.
What to remember
Specify useful brevity, reserve enough generation room, and tune effort on fixed cases. The winning policy preserves complete, correct outputs and reduces whole-task spending; a short truncated answer is a failed optimization.
Knowledge check
A briefing service on the Claude API returns verbose summaries. To cut output cost, an engineer lowers max_tokens from 8,000 to 600. Now 7% of briefs end with stop_reason: "max_tokens", and output spending barely falls. On the failing cases, the required final brief alone needs more than 600 tokens. What should the architect change?
max_tokens is a hard ceiling the model cannot see, so lowering it does not make Claude write less — it cuts off the responses that needed the room, and those turns are still billed. Brevity comes from what Claude is asked for: a format or schema with only the needed content, and effort. A max_tokens stop is a failed attempt. Lower effort still needs enough headroom and concise instructions; restoring headroom alone leaves the verbose format unchanged.A contract-extraction pipeline uses structured outputs. It retries any response whose JSON fails to parse, raising max_tokens each time. Logs show that some failing responses end with stop_reason: "refusal" and others with stop_reason: "max_tokens". How should the retry policy change?
max_tokens on retry for both stop reasonsmax_tokens on retry only for max_tokens stopsmax_tokens stop means the object ran out of generation room, and the documented remedy is a retry with a higher max_tokens . A refusal takes precedence over schema constraints, so neither more room nor a stricter schema changes it; it needs its own handling path.A team sweeps output_config.effort across low, medium and high on a fixed case set. To save money during the sweep, they also lower max_tokens to 2,000 for the high runs only. The high runs show many stop_reason: "max_tokens" failures and lose to medium. What conclusion does this result support?
high effort is worse for this workload and can be ruled outhigh with the same generous caphigh effort needs an added instruction telling Claude to stay briefmedium wins because effort caps its reasoning below 2,000 tokensmedium under any particular count.References
- Anthropic, Reducing latency
- Anthropic, Structured outputs
- Anthropic, Effort
- Anthropic, Optimizing for cost and intelligence
- Anthropic, Extended thinking