Skip to lesson

AnthropicClaude Certified Architect – Professional2026-104.5 Optimize token usage, latency, and cost-performance trade-offs

Baseline spending before choosing an optimization

Completed by the knowledge check

Learning objective

Establish per-task cost and latency baselines — including cache, reasoning, tool-result, and retry tokens — and validate every optimization against quality.

Core idea

A smaller response bill does not prove that a workflow became cheaper. The change might remove required context, create another tool round trip, or increase regeneration after incomplete answers. You need a baseline that connects every request to the business task it serves and scores the resulting outcome.

Anthropic’s cost-optimization guidance recommends measuring production-weighted tasks, comparing score against spending, and testing the winner in shadow before cutover. This lesson turns that method into an optimization baseline. See 1.6.2 for the whole workflow cost ledger and 4.1.3 for defining user-facing performance measures; here you identify which token and waiting-time components a proposed change actually moves.

How it works

Freeze a comparable task configuration

Record the model, deployment platform, prompt revision, tool definitions, retrieval configuration, max_tokens, and output_config.effort beside each run. Separate interactive and deferred tasks: the same document analysis can serve a person waiting for an answer or tomorrow’s report. Their latency acceptance criteria differ even if their quality rubric is identical.

Use a fixed evaluation set weighted like production, with identifiable difficult cases. Keep input documents, expected outcomes, and grading criteria constant between baseline and candidate. Repeated runs help expose variation; a candidate should not win because it received an easier sample. Define the acceptable quality loss before examining the cost results, including any categories where no loss is acceptable.

The baseline is an experiment record, not merely a dashboard average. Preserve task identity across retrieval, model calls, tools, retries, and final grading. A request that returns successfully but produces an unusable answer remains part of that task’s spending.

Reconcile usage without double counting

Price the five usage counts as 1.6.2 does; for this baseline, total processed input is usage.input_tokens plus cache writes plus cache reads. Counting uncached input alone understates the context processed by a cached request.

Reasoning is a subset of output spending. usage.output_tokens_details.thinking_tokens reports billed reasoning tokens; do not add them to output_tokens a second time. In streaming responses, that breakdown appears on the final message_delta. Count billed usage rather than estimating reasoning from the text displayed to the user.

A tool-result token is an attribution category within input, not a separate billing field. Track how much tool_result.content you append and how often subsequent requests replay it. A result can be uncached input when introduced and cached input later. Adding a separate “tool-result charge” to priced input would count those tokens twice. Tool-service fees, where applicable, belong in a separate ledger entry.

Count repetition and measure the complete wait

Associate every retry and quality-driven regeneration with the original task. Sum the usage returned across its requests. An attempt without a usable response is not proof of zero spending; reconcile the per-task sums against the aggregate reported by the Usage and Cost API rather than inventing zero-token records. Keep unresolved differences visible in the experiment record: a candidate with missing usage observations should not win because its measured spend is incomplete.

Use POST /v1/messages/count_tokens for input forecasts, not as the final invoice. Its estimate can differ from actual usage. The endpoint also rejects several inputs accepted by Messages, including most server tools and MCP connector requests; for those paths, use actual Messages usage. The forecast is useful for comparing candidate context shapes before generation, while actual usage settles the experiment.

Measure time to first token separately from time to an accepted result. Also retain retrieval time, tool waits, model calls, and retry delays. Streaming can improve perceived responsiveness while leaving task completion slow. Compare median and tail latency by task class; a lower average can conceal a regression in difficult requests.

Key distinctions and trade-offs

A token baseline explains the bill; a task baseline explains whether the spending was worthwhile. Keep both cost per submitted task and cost per accepted task. The latter divides all measured spending by accepted outcomes, including spending on tasks that failed.

Caching changes token prices and processing behavior; trimming changes what enters context. Lower effort can change reasoning, tool calls, and answers together. Change one lever at a time before testing a combined configuration, or you cannot explain whether savings came from reuse, less work, or lost quality.

Warm-cache measurements can flatter a candidate that receives little prefix reuse in production. Include cold starts, realistic gaps, and prompt revisions. Similarly, short successful sessions cannot represent a workload whose expensive cases require several repairs.

Architecture / decision scenario

A financial-services document-review team has a 30-second p95 completion target and requires at least 95% accepted reviews, with no regression on material-risk omissions. The platform architect compares two configurations on the same 1,000 cases. These are illustrative measurements, not Claude price quotes.

The baseline costs $100 across all attempts, accepts 960 cases, and finishes at 27 seconds p95. A heavily trimmed candidate costs $78, accepts 870, and finishes at 21 seconds. A cache-preserving candidate costs $86, accepts 959, and finishes at 24 seconds. The first candidate looks attractive on cost per task but violates the acceptance floor before economics can justify it.

The architect selects the cache-preserving candidate for shadow testing after reviewing the material-risk slice. Its record includes cache reads and both write durations, the reasoning subset of output, tool-result volume, and repeated requests. If the saving exists only under a warm cache, the recommendation must account for that production exposure. The risk-review owner signs off on quality; the platform owner owns the cost and latency comparison and rollback trigger.

Common exam traps

  • Adding reasoning tokens to an output total that already includes them.
  • Adding tool-result token estimates as a second input charge.
  • Counting only uncached input or the final successful request.
  • Treating token-count forecasts as exact billed usage.
  • Declaring a faster first token to be faster accepted completion.
  • Selecting the cheapest candidate before enforcing the quality threshold.

What to remember

Freeze comparable tasks, reconcile disjoint usage fields, and retain every attempt. Price reasoning inside output and tool results inside input. An optimization earns release only when its complete task cost and latency improve while the agreed quality constraints hold.

Knowledge check

References