Cut input without losing decisive context

Learning objective

Reduce input tokens with caching, context trimming, retrieval tuning, and tool-result pruning.

Core idea

Keep tool results limited to decision-relevant information while preserving identifiers, qualifications, and failures. The architectural decision is how that compact result interacts with retrieval, replayed history, and caching across the whole task.

Prompt caching lowers the price of repeated input; it does not remove that input from context. Retrieval and trimming change which information the model receives. Choose a combination that reduces task spending while preserving access to decisive facts, rather than optimizing the smallest payload or highest cache-hit rate in isolation.

How it works

Separate reuse from removal

Assign each input component a reason to remain: standing instructions, tool definitions, reusable reference material, task-specific passages, or active conversation state. Then classify it as stable and reusable, necessary but variable, or unnecessary for this decision. This component inventory gives you a place to explain every removal.

Use cache_control for stable prefixes. See 2.5.1 for pricing and prefix design. For this optimization, compare the measured usage.cache_read_input_tokens and usage.cache_creation_input_tokens before and after a context change. A shorter request can still cost more if it repeatedly rewrites a prefix that previously received cheap reads.

Confirm caching through usage, as in 2.5.1: a marker below the model’s minimum produces no cache entry, while a route that rejects top-level cache_control needs a supported request shape before you can measure any saving.

Trim at a meaningful task boundary

Remove duplicated instructions, unused tool definitions, and irrelevant history before weakening task evidence. The owners of general context selection and compaction are 2.4.2 and 2.4.3; here the choice is whether their net savings justify the information loss and cache disruption for this workload.

Anthropic’s cost guidance measured context editing costing more than it saved on the short, twenty-issue run; context levers pay only on a session long enough to need them. A context transformation can shorten later requests but also add processing and invalidate reusable prefixes. Measure the entire remaining task, including the transformation, rather than reporting only the first shorter request.

Keep the stable prefix ahead of variable content. In particular, a changing timestamp before shared instructions defeats exact-prefix reuse. Make cache-invalidating prompt or tool changes at a natural task boundary when possible, then measure reads again. Treat prompt revision and context policy as one release: an unexplained cache-read collapse after trimming is an optimization regression worth investigating.

Reduce retrieval breadth only after measuring coverage

Distinguish candidate retrieval from passages admitted to the model. You can retrieve a broad candidate set, rerank it, and send a smaller relevant selection. Anthropic’s Contextual Retrieval article combines semantic retrieval with BM25 and discusses reranking; it also notes that reranking adds runtime cost and latency. See 3.6.3 for the retrieval stages and 3.5.5 for their quality measures.

For this decision, record candidate count, admitted count, admitted tokens, retrieval failures, and answer quality together. Those counts are application configuration, not universal Messages API fields. Test a range instead of assuming that fewer passages are always better. Anthropic’s reported experiments found that twenty admitted chunks worked better than five or ten; that is a warning against blind reduction, not a required value for every corpus.

Preserve qualifications and source provenance when trimming a passage. A short extract that loses an exception or effective date may be semantically different from its source. Evaluate questions that need several documents, not only questions whose answer lives in the first retrieved paragraph. Input reduction must retain enough context to support justified abstention as well as correct answers.

Prune tool results without creating retrieval loops

Prune a result before it enters history. On Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, rewriting an earlier tool_result client-side invalidates every thinking block after it; use server-side context editing or compaction instead, which preserve those thinking blocks (see 2.4.3).

The professional cost question extends beyond shortening one response: does the compact view force repeated lookups to recover omitted information? Track repeated calls and downstream failures beside the input reduction. Anthropic’s tool-design guidance recommends pagination, range selection, filtering, and truncation with sensible defaults. Expose those controls in the tool’s schema so a follow-up can ask for a targeted range instead of downloading the entire result again.

Version the compact view with the retrieval policy in the experiment record. Include cases where the first lookup is sufficient, where a targeted follow-up is necessary, and where the tool fails. Compare total admitted tokens across those paths. A candidate that saves on ordinary responses but repeatedly reloads details for difficult cases needs a traffic-weighted comparison before release.

Key distinctions and trade-offs

Use a comparison record that separates the four levers:

Do not change all four first and then attempt to explain the result. Compare each candidate with the 4.5.1 baseline, followed by the intended combination. A retrieval change that helps alone may hurt after tool pruning removes the identifiers needed for follow-up. The integration test is task completion, not the sum of separate token-saving percentages.

Architecture / decision scenario

A retail support platform must answer policy questions within eight seconds p95 and retain at least 96% rubric acceptance. Its architect proposes replacing a twenty-passage response with five passages, shortening order-tool results, and caching stable policy instructions. The policy owner requires correct handling of sale exclusions and effective dates.

On 600 fixed cases, the five-passage candidate reduces admitted retrieval tokens by 60% but acceptance falls from 97% to 91%. A reranked ten-passage candidate plus compact order results reaches 97% at seven seconds p95, including reranker time. The team also removes a per-request timestamp from before the cached prefix; actual usage confirms reads after the initial write. These are illustrative workload measurements.

The architect rejects five passages despite its larger token reduction and selects the combined candidate for shadow testing. The tool owner checks whether detailed follow-ups increase, the retrieval owner tracks missing sale exclusions, and the platform owner watches cache writes after prompt revisions. If one component regresses, their comparison record identifies which change to revert.

Common exam traps

• Claiming caching reduces the number of context tokens.
• Treating cache_control as proof that a short prefix was cached.
• Deleting passages before measuring retrieval and answer coverage.
• Reporting a shorter history without including cache rewrite costs.
• Shrinking a tool result while ignoring extra detail-fetch turns.
• Treating an article’s successful chunk count as a universal setting.

What to remember

Optimize necessary information and reusable prefixes together. Cache stable material, trim irrelevant state, tune admitted retrieval, and keep tool views sufficient for completion. Release the combination only after measuring whole-task spending, latency, and quality.

Knowledge check

References

• Anthropic, Prompt caching
• Anthropic, Optimizing for cost and intelligence
• Anthropic, Introducing Contextual Retrieval
• Anthropic, Writing effective tools for agents — with agents