AnthropicClaude Certified Architect – Professional2026-103.8 Evaluate progressive discovery vs. monolithic context strategy
Measure what discovery actually saves
Completed by the knowledge check
Learning objective
Weigh up-front token cost and cache stability against discovery latency, extra round trips, stale catalogs, and missed context.
Core idea
Progressive discovery exchanges context loaded before work begins for acquisition steps during the work. A smaller initial request is a useful measurement, but it is not proof of a cheaper or faster accepted outcome.
Measure the full path: cache writes and reads, uncached input, generated searches, fetched detail, repeated inference, and failures that require another attempt. A strategy that saves input tokens can still miss the latency target or omit the document required for a correct answer. Compare monolithic, progressive, and hybrid candidates on the same representative cases.
How it works
Count the complete input
For each Messages response, collect usage.input_tokens, usage.cache_creation_input_tokens, and usage.cache_read_input_tokens. Their sum is total input processed for that request. input_tokens alone omits tokens read from or written to the cache, so comparing that field between cached and uncached candidates understates one side.
Sum those categories across the complete workflow, and retain usage.output_tokens as a separate cost component. A discovered definition can remain available in subsequent conversation history, and a fetched procedure can be replayed through later turns. Deferred does not mean paid for once and then absent.
Price each category with the selected model’s documented multipliers; see 2.5.1. Tool search preserves the prefix while loading detail inline, whereas rebuilding the tool definitions per request can break reuse; see 3.1.3.
Compare reuse across the workload
A larger monolithic prefix can have a lower effective cost than its raw size suggests when one write serves many warm reads. Evaluate the requests that actually share that prefix, not an assumed organisation-wide hit rate. A progressive candidate starts smaller but can accumulate discovered definitions and retrieved content across later turns. Compare its entire sequence against the monolith’s write-and-read sequence.
For an illustrative ten-request comparison, suppose the monolithic candidate writes a 40,000-token prefix once and reads it on nine requests. The progressive candidate writes a 6,000-token core once and reads it nine times, but its investigations add 50,000 uncached input tokens across the sequence. Compare the first candidate’s 40,000 write tokens plus 360,000 read tokens with the second’s 6,000 write tokens, 54,000 read tokens, and 50,000 uncached tokens, before adding output and retrieval costs. The smaller starting prefix alone cannot settle the result.
Repeat the comparison with the observed change cadence. If policy or definition updates repeatedly replace the monolith’s shared prefix, the expected warm reads may never occur. Conversely, if most requests discover the same detail, that material is a candidate for the preloaded core. Measure cold starts, ordinary reuse, and release-day invalidation as separate workload slices rather than averaging away the reason a strategy wins.
Separate steps from network round trips
Server-side tool search runs inside an API response. Its query generation, search, and result interpretation consume work, but it is not automatically another client-side HTTP round trip. In contrast, a client tool that fetches a runbook normally requires your application to execute it and send its tool_result on a subsequent Messages request.
Measure both model-facing steps and end-to-end elapsed time. Record time spent searching, waiting on external services, and producing the accepted answer. Anthropic’s context-engineering guidance warns that runtime exploration can be slower and can chase dead ends. Median latency can look healthy when most requests use the preloaded core while tail requests accumulate serial fetches.
Evaluate p95 for the whole workload and for discovery-dependent cases. Those distributions answer different questions: whether the service meets its overall target, and whether specialist users bear an unacceptable delay. See 1.6.3 for deriving service targets from user journeys.
Treat freshness as a measured dependency
For each catalog-dependent case, record whether stale metadata sent the workflow toward the wrong resource, delayed acquisition, or prevented an accepted answer. Those outcomes make freshness part of the strategy comparison rather than a separate availability statistic.
MCP revision 2026-07-28 defines ttlMs and cacheScope on cacheable list and read results. The former is a freshness hint in milliseconds; the latter distinguishes "public" and "private" caching scope. These fields help the host manage acquisition reuse, not Anthropic prompt-cache reuse. Do not treat their presence as proof that a resource’s content cannot change.
Automated Claude Code and SDK sessions refresh only tools on list_changed, so measure resource reacquisition in the automated path rather than assuming interactive refresh covers it.
Measure the context that never arrives
See 3.1.3 for tool discovery recall. Extend the required-item labels to documents and resources: record whether each necessary source loaded before the decision, then separate misses caused by selection from misses caused by stale catalog entries or inaccessible content. This extension is an application evaluation measure, not an Anthropic API metric. A tool can be discovered correctly while the procedure it needs remains unread.
Tool search represents a no-match result with an empty tool_references array. Record that outcome rather than interpreting it as proof that the capability does not exist. Test synonyms, ambiguous requests, renamed entries, and expired catalogs. A resource retrieval error likewise differs from an authoritative statement that the source has no relevant rule.
Key distinctions and trade-offs
Compare the candidates with these measures:
| Measure | What it resolves |
|---|---|
| Total workflow token categories | Whether the initial reduction survives later turns |
| Cold and warm cache usage | Whether the reusable prefix actually reuses |
| Overall and discovery-case p95 | Whether the tail violates its response target |
| Required-context recall | Whether savings come from omitting necessary material |
| Stale or inaccessible source outcomes | Whether acquisition remains trustworthy |
Only compare costs at an acceptable quality level. A cheap answer based on an unfetched exception is not an accepted outcome.
Architecture / decision scenario
A technology company’s incident assistant has a ten-second p95 target. In an illustrative held-out evaluation, the cached monolithic candidate carries 42,000 input tokens and reaches 7.2 seconds p95. A fully progressive candidate starts with 6,000 tokens, but reaches 12.8 seconds p95 and misses required context in 8% of cases. The hybrid reaches 8.6 seconds p95 and misses required context in 1% of cases.
The tempting decision is to ship the progressive candidate because its starting input is smallest. The architect instead compares complete workflow usage and rejects it on both latency and coverage. The hybrid is the candidate for further rollout, subject to the team’s stated quality acceptance threshold; a 1% miss rate is not automatically acceptable for every incident class.
The platform owner preloads the most common diagnostic procedure, retains specialist detail behind discovery, and tests a deliberately stale resource catalog in the automated runtime. The operations owner labels the required context for each incident class. Together they verify that refreshing resources and improving descriptions repairs misses without pushing specialist p95 beyond the agreed budget.
Common exam traps
- Comparing only
usage.input_tokenswhen one candidate uses caching. - Equating a smaller first prompt with lower complete-workflow cost.
- Counting every server-side search as a client network round trip.
- Reporting median latency while discovery-dependent p95 violates the target.
- Treating empty search results as authoritative absence.
- Confusing MCP cache freshness with prompt-cache matching.
Where the exam guide differs
The exam guide names no MCP revision and was written while revision 2025-11-25 was published. This Bite cites revision 2026-07-28, a later revision that adds required ttlMs and cacheScope fields to cacheable catalog and resource-read results and replaces resource subscription methods with subscriptions/listen. An exam item can express freshness without these fields; keep the freshness responsibility, and implement the fields and notification path supported by the negotiated revision.
What to remember
Discovery saves only what complete accepted workflows demonstrate. Count every token category, distinguish server steps from external round trips, inspect tail latency, and test both stale catalogs and context that never loads.
Knowledge check
Two loading candidates run on the same cases. The cached monolithic candidate reports 1,800 usage.input_tokens per request; the progressive candidate, which uses no caching, reports 9,000. A dashboard concludes that the monolith processes less input. What should the comparison count instead?
input_tokens excludes tokens written to or read from the cache, so it understates the cached candidate. Total input is the sum of input_tokens , cache_creation_input_tokens and cache_read_input_tokens , counted across every turn, because discovered and fetched material can be replayed in later requests. Output tokens are a separate cost component, not a correction to input.An incident assistant uses server-side tool search for its deferred tools and a client tool, fetch_runbook, that the application executes. To estimate added latency, the team must count the extra Messages requests the application sends. Which step requires one?
fetch_runbook call whose result the application returnstool_reference in later turnstools arraytool_result on a subsequent Messages request. The API expands tool_reference blocks itself, and a deferred definition sits in the same request rather than adding one.An assistant’s overall p95 latency is 8.1 seconds against a 10-second target. Discovery-dependent cases are 4% of traffic, and their own p95 is 15 seconds. Specialist users who depend on those cases are held to the same 10-second target. What should the evaluation conclude?
During evaluation, tool search returns an empty tool_references array for a request about reversing a chargeback, although the approved catalog contains dispute_refund_undo for exactly that operation. How should the evaluation record the outcome?
tool_references array rather than an error, and it is not an authoritative statement that the capability is absent. Here the required tool exists, so the outcome is a missed required item; testing synonyms and the tool’s description finds whether its searchable wording failed.References
- Anthropic, Prompt caching
- Anthropic, Tool search tool
- Anthropic, Effective context engineering for AI agents
- Anthropic, Connect Claude Code to tools via MCP
- Model Context Protocol, Resources, revision 2026-07-28
- Model Context Protocol, Changelog, revision 2026-07-28