AnthropicClaude Certified Architect – Professional2026-103.3 Evaluate accuracy-latency trade-offs and justify configuration decisions
Match configuration levers to measured errors
Completed by the knowledge check
Learning objective
Identify configuration levers that move accuracy and latency: model tier, reasoning effort, context size, retrieval depth, tool calls, and agent count
Core idea
For an architect configuring a production Claude integration, accuracy and latency belong to the complete configuration. A model benchmark does not tell you whether the application retrieves the deciding policy, waits for three unnecessary tool calls, or spends its entire response budget on elaboration. Identify the error’s cause before buying more computation.
Anthropic recommends establishing a prompt that performs well before optimizing latency. That baseline makes the cost of each reduction visible. The useful question is which mechanism earns its place on the critical path, rather than which setting is largest. See 2.1.1 and 2.1.3 for model selection and reasoning controls themselves; this lesson compares their effects with the integration’s other levers.
How it works
Separate six configuration choices
Record the deployed model, output_config.effort where supported, the supplied system and messages content, retrieval settings, tool schedule, and delegation plan. These are different decisions even when one deployment template stores them together. Retrieval depth and agent count are integration choices, rather than additional fields on a Messages request.
The six levers address different failure patterns:
| Lever | Error it might repair | Latency exposure |
|---|---|---|
| Model tier | Supplied evidence requires reasoning the selected tier fails to perform | Different model processing and generation time |
| Reasoning effort | The model skips necessary analysis of interacting constraints | More response computation, including thinking when active |
| Context size | Required instructions or evidence were absent from the request | More input processing; additional irrelevant material can dilute useful context |
| Retrieval depth | The deciding document was not among retrieved candidates | More search or reranking work and potentially more supplied chunks |
| Tool calls | A decision needs an external fact or calculation | Execution time plus model continuation round trips |
| Agent count | Independent investigations exceed one agent’s practical scope | Dispatch, separate inference, completion waits, and synthesis |
The table gives hypotheses, not guaranteed improvements. Increasing any quantity without diagnosing the failure can add time without changing the answer. Measure the candidate on the same cases; see 3.3.4.
Connect effort to the whole response
Use the effort mechanism from 2.1.3 as a workload-specific latency lever, rather than applying one setting to every task. Claude Sonnet 5.5 supports five effort levels: low, medium, high, xhigh, and max. Anthropic recommends starting at medium or low for chat and other latency-sensitive work, and using xhigh or max only where evaluations show a quality gain. On Claude Haiku 5.5, low is the fastest and cheapest effort level, but long agent prompts are more likely to skip a search, stop early, or skip a check at that setting. Inspect whether required retrieval and verification still happen, rather than measuring only the shorter response.
Anthropic’s latency guidance names Claude Haiku 5.5 as its speed-critical model choice, with effort as the main speed control. If the task needs an Opus tier, Fast mode keeps the same model and offers up to 2.5 times higher output tokens per second at premium pricing: set speed: "fast" with anthropic-beta: fast-mode-2026-02-01. It is a research preview requiring requested access, supported on Claude Opus 5.5, Opus 5, and Opus 4.8 on the Claude API, including Managed Agents, only; it is unavailable on Amazon Bedrock, Claude Platform on AWS, Google Cloud, and Microsoft Foundry. Its speed benefit targets output generation, not time to first token, so it does not repair a slow retrieval stage.
A low-effort configuration that preserves simple classification accuracy can still fail on interacting exceptions. Compare those cases separately rather than concluding from a high aggregate score that thoroughness was unnecessary. Model choice and effort interact: one tier’s low setting is not a numerical substitute for another tier’s high setting.
Distinguish retrieval work from context volume
Retrieval depth determines how many candidates the search pipeline considers. Context size describes what actually reaches Claude, including policy, examples, retrieved passages, and conversation history. Increasing one does not require increasing the other. You can retrieve a wider candidate set, rerank it, and supply a smaller selected set.
Anthropic’s Contextual Retrieval account explains that reranking adds a runtime step and that reranking more chunks trades latency and cost for retrieval performance. It is an architectural option to test, not a fixed recommendation to send every candidate. The candidate count and retained count are settings in your retrieval implementation; document both in the configuration record.
If the correct policy is absent from all supplied passages, increasing output_config.effort addresses the wrong stage. If the passage is supplied but the answer mishandles an exception, deeper retrieval can add irrelevant text while leaving the reasoning defect intact. Separate missing-document failures from misuse-of-document failures before changing either setting. Chunking and retrieval design belong to 3.5 and 3.6.
Account for calls and agents separately
In the Messages API, several tool_use blocks can appear in one response. The application decides whether their executions overlap. Count sequential response-and-result cycles as well as individual calls: five independent reads in one dispatch have a different critical path from five dependent reads across five turns. Use 3.3.2 to remove unnecessary serialization.
Adding agents is a wider intervention. Anthropic’s multi-agent research account describes independent contexts and parallel exploration, but also coordination costs and poor fit for work requiring tightly shared context. A delegation plan that duplicates investigation and then waits for synthesis can be slower than one scoped agent. More agents can improve coverage only when their assignments add distinct information worth the extra work.
Key distinctions and trade-offs
A speed setting and an accuracy setting are rarely independent. Lower effort can change tool behavior; broader retrieval changes the input; additional agents alter both coverage and synthesis requirements. Treat a configuration as a reproducible combination rather than ranking settings in isolation.
Tool latency is also not model latency. Moving a lookup off the sequential path can shorten completion while retaining its result. Switching to a faster tier might leave the slow lookup untouched and reduce reasoning quality. Prefer the change that addresses the measured bottleneck, then verify that accepted-answer accuracy holds.
Architecture / decision scenario
A retail returns assistant must complete recommendations within six seconds at the 95th percentile. In an illustrative replay, a compact tier with four supplied passages scores 90% correct at three seconds. A stronger tier with high effort, twenty passages, two verification turns, and four agents scores 97% at eleven seconds. Neither configuration meets the team’s combined requirement of at least 96% accuracy and the six-second limit.
The tempting proposal is to retain the second configuration and stream it. The integration architect rejects that as the completion-time remedy: the required checks still precede acceptance. Error review shows missing seasonal exclusions in the compact configuration and redundant investigation in the larger one.
The architect proposes wider retrieval reranked to six passages, a supported model at medium effort, one inventory tool dispatch, and a single agent. This exercises all six levers: model and effort are explicit, supplied context is bounded, candidate retrieval is wider, tool round trips are reduced, and agent count removes duplicate work. The returns-policy owner grades exclusion cases while the platform owner measures completion under store-opening load. The proposal ships only if that run meets both targets; the table’s pilot results do not prove the proposed combination works.
Common exam traps
- Raising effort when the deciding source never reaches the model.
- Equating retrieved candidate count with supplied context size.
- Assuming lower effort changes only thinking, leaving tool behavior identical.
- Counting tool calls without counting sequential model round trips.
- Adding agents to work that requires tightly shared state without measuring synthesis overhead.
- Selecting the fastest model while ignoring the accuracy requirement.
What to remember
Diagnose the failed stage, name the corresponding configuration choice, and measure the complete combination. Model tier, effort, context, retrieval, tools, and agents all move the accuracy-latency result. Spend time where it repairs the observed failure, and remove work that does not earn its delay.
Knowledge check
An insurance claims assistant on the Claude API supplies Claude with the top five chunks its policy search returns. Error review shows that in most failed cases the governing endorsement was in none of the five supplied chunks, and that answers are correct whenever it is supplied. The route has about one second of spare latency budget. Which change targets the measured failure?
output_config.effort from medium to high on every claimA contract-drafting service on the Claude API runs Claude Opus 5.5 and has been granted research-preview access to Fast mode. Traces show that time to first token is short and stable, and that most of the p95 completion time is spent generating a long clause schedule the customer requires. Evaluations show that a smaller model tier and a lower effort level both drop accuracy below the floor. Which change should the team test first?
output_config.effort to low for the clause schedulemedium effortspeed: "fast" and the fast-mode-2026-02-01 beta header, keeps the same Opus model and raises output tokens per second, which is where this route spends its time; its benefit targets output generation, not time to first token, which is already short here. The evaluations rule out the smaller tier and the lower effort level, and cache pre-warming removes prefix processing before the first token rather than shortening a long generated output.A retail operations agent runs on Claude Haiku 5.5. After the team moved it from medium to low effort, median latency fell and the aggregate accuracy score held. A per-case review of long refund-exception tasks shows the agent sometimes answered without running the required policy search; those responses ended with end_turn, and the policy library is far larger than the context window. The latency target is still met at medium. What should the team do?
low and track the aggregate accuracy scoremax_tokens so the agent has room to finish the searchlowmedium and re-check the exception caseslow effort Claude Haiku 5.5 is more likely to skip a search, stop early or skip a check in long agent prompts, so a lower setting can change tool behavior and not only response length. A held aggregate score hides the exception segment, the responses were not truncated, and the library cannot fit in context; returning that workload to medium , which still meets the latency target, and measuring the exception cases addresses the observed failure.References
- Anthropic, Reducing latency
- Anthropic, Effort
- Anthropic, Introducing Contextual Retrieval
- Anthropic, Parallel tool use
- Anthropic, How we built our multi-agent research system
- Anthropic, Fast mode (research preview)