Select context for the next decision

Learning objective

Curate and order context to the smallest high-signal set, accounting for context rot, stale evidence, and lost-in-the-middle effects.

Core idea

Compact tool responses, labeled documents, and fresh observations remain necessary; at system scale, decide what the whole request admits. Give the request builder a selection policy that preserves the facts needed for this decision across instructions, tools, documents, and history, with an owner for changes to that policy.

Anthropic calls for the smallest set of high-signal tokens that supports the desired outcome. “Smallest” is constrained by sufficiency: a shorter request that drops the only exception to a policy is a worse design. The operating question is whether each included passage earns its cost and whether each excluded passage remains recoverable when needed.

How it works

Select by decision dependency

Start from the question the agent must answer and the action it might recommend. Identify which observations distinguish the possible outcomes. For a retail return, the product variant, purchase date, applicable policy revision, and any documented exception can matter more than the complete order history. These are application selection criteria, not Anthropic retrieval fields.

Apply that selection across the request. Keep broadly applicable instructions in system, task capabilities in tools, and the selected observations in messages. A concise document set beside a sprawling tool catalog or repetitive transcript still wastes the shared budget. See 2.4.1 for allocating those components and 3.8 for progressive capability discovery.

Anthropic’s context-engineering article describes just-in-time access: keep lightweight identifiers such as paths, stored queries, and links, then retrieve detail when needed. Adopt that approach for material whose relevance is uncertain. Keep the policy section that determines eligibility in active context; retain a source pointer to the rest instead of importing the whole policy library. Define the follow-up tool route before excluding detail, so the agent has a way to resolve uncertainty.

Just-in-time retrieval spends additional tool turns and depends on source availability. Upfront inclusion is reasonable when the next decision predictably needs the same small reference set or when fetching it would breach the service target. The builder should make this choice from measured task behavior, not use a universal rule that all data must arrive late.

Separate three different failures

Use the assembled request and its source versions to diagnose what failed:

Context rot means accuracy and recall can degrade as token count grows. A larger window permits more content; it does not establish that all of that content helps. Compare a sufficient selected request against the expanded request on the same decisions, including contradictory and near-duplicate sources.

Freshness is a separate dimension. A short excerpt of yesterday’s policy can be precisely quoted and still be unsuitable for today’s decision. Preserve the source revision or observation time with the excerpt. Before a consequential action, the application can require a fresh lookup of the governing source and replace the selected observation with its result. That requirement belongs in the tool-backed workflow; asking Claude to be accurate does not update a document.

Lost-in-the-middle concerns information present but overlooked. Inspect inclusion before tuning position. If selection or summarization removed the exception entirely, ordering cannot restore it. If it is present, test layouts that give it clear prominence without suppressing competing observations. This diagnostic distinction prevents a formatting fix from hiding an upstream selection defect.

Treat ordering as a tested policy

Start from the established long-input layout and opening findings. At system scale, make the request builder’s layout a versioned policy: record the selected source revisions and the assembled input used for each evaluation so you can reproduce which material occupied each position.

A policy release can change which sources are selected, how they are summarized, and where they appear. Require a comparison that isolates the claimed benefit before adopting the release. If a shorter layout trial also excludes an obsolete source, the result supports that combined change; it does not establish that ordering alone fixed recall. Keep the previously accepted assembly available for comparison when a later source revision exposes an omission.

Evaluate selection and ordering separately. Hold the document set fixed while changing layout, then hold layout fixed while changing selection. Include cases where decisive material originates near the middle, an obsolete revision looks nearly identical, or the smallest excerpt omits a qualifying sentence. Compare supported decision accuracy, source attribution, input tokens, and latency. These measurements let the policy owner defend a change without attributing every improvement to shorter context.

Key distinctions and trade-offs

Sufficiency versus exhaustiveness: Include what distinguishes outcomes; retain access to the remaining material. Exhaustive inclusion can introduce irrelevant competitors.

Freshness versus visibility: Promoting stale evidence makes it more visible, not more valid. Refreshing evidence does not guarantee the model uses it correctly.

Ordering versus restoration: Layout helps the use of supplied facts. Recovery from omitted facts requires changing the upstream selection or summary.

The request-builder owner manages selection and layout. Source owners define revision and freshness semantics, while the product owner accepts any quality or latency trade-off. This division matters when “send less” conflicts with a requirement to consider exceptions: preserve the exception and remove unrelated bulk first.

Architecture / decision scenario

A retail service must answer return questions within eight seconds and use the policy applicable to the purchased product. An illustrative replay shows 96% supported decisions at six seconds with a 24,000-token selected input, against 89% at eleven seconds when the builder adds all regional policies and full history. One disputed case depends on a safety-recall exception buried in the fifth document; another uses a superseded regional rule.

The proposed alternative is a larger window plus “read all policies carefully.” The architect rejects it because it retains both the distracting variants and the obsolete rule. The builder selects the purchased variant and governing revision, retains the recall exception and source pointer in the opening findings, and puts the comparison question after the labeled documents. Before recommending the refund, a policy lookup refreshes the governing revision.

The service owner accepts the design only after separate selection and layout trials preserve the recall exception and resolve the stale-rule case. If a question needs an excluded appendix, the agent retrieves it by its source identifier instead of inferring its contents. The team’s target is the smallest sufficient set for the decision, with an explicit path to more detail.

Common exam traps

Avoid these curation mistakes:

• Equating fitting the window with helping the answer.
• Choosing the shortest excerpt even when it removes a qualification.
• Trying to repair stale observations through ordering.
• Reformatting a request whose decisive evidence was never included.
• Attributing a quality gain to ordering when the comparison also changed selection.
• Promising that a particular position eliminates all omissions.

What to remember

Select context by what changes the decision, retain recoverable source pointers, and inspect freshness independently of volume. Test ordering with the same selected evidence before attributing quality gains to layout.

Knowledge check

References

• Anthropic, Effective context engineering for AI agents
• Anthropic, Context windows
• Anthropic, Prompting best practices