Repair evidence gaps before accepting synthesis

Learning objective

Evaluate synthesis for coverage gaps, re-delegate targeted investigations, and re-synthesize until findings are sufficient.

Core idea

For an architect designing a research coordinator, a fluent synthesis is a candidate answer, not proof of complete coverage. Compare it with the original request and the evidence gathered for each requirement. If a consequential gap remains, identify the missing work, delegate a targeted investigation, and synthesize again using the revised evidence.

Anthropic’s research architecture follows this loop: the lead researcher combines subagent findings and decides whether to gather more information. Its evaluator-optimizer pattern adds a useful distinction: evaluation gives specific feedback against clear criteria, rather than asking for an unspecified improvement. The coordinator must turn that feedback into work that can change the answer.

How it works

Define sufficiency before reviewing prose

Use the requirement-to-evidence matrix from 1.2.2 as the evaluation baseline. Specify what makes the answer useful: each requested criterion addressed, material claims supported, alternatives compared consistently, and unresolved uncertainty stated. Define these as application acceptance criteria.

Anthropic’s research-system evaluation rubric considers factual accuracy, citation accuracy, completeness, source quality, and tool efficiency. Those dimensions help separate answer quality from surface polish. A beautifully written report can fail completeness; a well-sourced paragraph can still cite evidence that does not support its specific conclusion.

Evaluate the answer against the original request, not only against the coordinator’s decomposition. If the first plan accidentally omitted data handling, evaluating whether all planned tasks finished preserves the omission. The user’s question remains the authority for required coverage even after several rounds of delegation.

Define which gaps are material. Missing support for a recommendation or an excluded alternative may require more work. A missing optional illustration may not. This prevents a loop that repeatedly expands the report without improving the decision it supports.

Diagnose the gap before choosing the follow-up

Classify why the synthesis is insufficient. Missing evidence needs retrieval; conflicting evidence needs reconciliation; incomplete reasoning needs analysis; a distorted summary needs synthesis correction. These defects can appear in the same paragraph, but they require different actions.

The following distinctions guide targeted delegation:

Treat these follow-up roles as application-defined. Define them through agents when specialization is useful, or let the coordinator perform the work itself as described in 1.2.3.

When a report contains evidence but the draft omitted it, another broad search is wasteful. When evidence is absent, rewriting the same draft cannot supply it. The diagnosis determines whether you need a search subagent, an analysis subagent, synthesis alone, or a sequence of those operations.

Delegate a query that can close the gap

The Agent SDK invokes specialists through the Agent tool. Put the exact unresolved criterion, relevant findings, source locations, and requested output in the invocation prompt. See 1.2.1 for the context boundary. “Investigate the gap” is insufficient unless you explicitly state what the gap is.

For example, a follow-up search might ask for the documented recovery behavior for one alternative under a specified failure condition. An analysis assignment can then compare that result with the shared workload assumptions and explain whether it changes the recommendation. Keep the surrounding decision visible while narrowing the local investigation.

The returned report should identify the obligation addressed, supporting evidence, conclusion, and remaining uncertainty. These are your output-contract fields. A worker that finds no evidence should return that outcome rather than silently replace the requested claim with an easier one. Negative retrieval results tell the coordinator what remains unresolved; they do not prove the underlying claim false.

When continuity matters, resume a custom or general-purpose investigator. Capture session_id from query messages and agentId from the Agent result. In the next query, set the resume option to that session ID and name the agent ID in the prompt. Keep the same custom definition in agents. Built-in Explore and Plan agents are one-shot and do not return the required agent ID.

Re-synthesize and inspect the changed answer

Merge accepted follow-up findings into the evidence set before generating the revised synthesis. Preserve unaffected findings and replace conclusions that the additional evidence invalidates. Appending a corrective footnote while leaving a conflicting recommendation intact is not a completed refinement.

If the application uses a synthesis specialist, invoke it again with the updated findings, original requirements, and explicit conflicts to resolve. If the coordinator synthesizes directly, perform the same update in its context. Evaluation then inspects the revised answer as a whole: does the repaired paragraph agree with the recommendation, assumptions, and stated limitations?

Keep an application-owned record of gaps, assignments, accepted evidence, and decisions. Preserve usable findings after failures; see 1.2.1 for routing interrupted worker outcomes.

Stop for evidence sufficiency or a stated boundary

A successful loop stops when material requirements meet the acceptance criteria. It does not continue until every imaginable detail has been researched. Conversely, a fixed number of rounds cannot establish completeness: the final round may still leave a critical claim unsupported.

Bound refinement with an application iteration policy and deadline. For SDK spend control, maxBudgetUsd in TypeScript and max_budget_usd in Python cap the query’s spend including subagents in the releases documented by the subagents page: TypeScript v0.3.219 and Python v0.2.127 or later, bundling Claude Code v2.1.219 onward. Reaching that cap produces the error_max_budget_usd result subtype, not evidence that the answer is complete. If the application stops before sufficiency, return an explicit partial result or defer the recommendation rather than lowering the rubric.

Refinement repeats targeted work until the evidence is sufficient or a stated boundary stops it.

Key distinctions and trade-offs

Evaluation versus synthesis: Synthesis combines findings into an answer. Evaluation checks whether that answer satisfies the request and follows from the evidence. One pass can do both poorly unless the criteria remain explicit.

Targeted refinement versus wholesale restart: Targeted work closes a diagnosed gap while preserving supported findings. A restart is appropriate only when the earlier framing or evidence set is fundamentally unusable.

Sufficiency versus certainty: A useful answer can contain explicit uncertainty. Unsupported confidence does not compensate for missing evidence, and bounded uncertainty does not always require another round.

Refinement improves coverage but adds time, cost, and opportunities for inconsistency. Require each follow-up to name the decision it could change. That keeps the loop focused on evidence rather than endless prose revision.

Architecture / decision scenario

A research coordinator drafts a recommendation for a customer-support agent deployment. Both alternatives have documented data-handling findings, but one resilience conclusion lacks supporting evidence. The draft also compares resource estimates based on different workload assumptions.

The evaluator flags those two defects against the original criteria. The coordinator sends a targeted recovery-behavior query to a search specialist and a workload-reconciliation assignment to an analysis specialist. It does not rerun the completed data-handling investigation. Both workers receive the exact criteria and relevant evidence references.

The follow-up evidence changes the preferred alternative under the stated workload. The coordinator updates the evidence set and invokes synthesis again, requiring the recommendation and limitations to reflect that change. A second evaluation checks coverage, citation support, and consistent assumptions across the whole answer. If a material gap persists, the coordinator repeats targeted work within its boundary; if the deadline or spend cap intervenes, it returns the unresolved decision explicitly. Completion means sufficient supported findings, not merely a finished synthesis call.

Common exam traps

• Accepting fluent synthesis without checking every original requirement.
• Asking all workers to repeat research instead of naming the missing evidence.
• Treating another rewrite as a remedy for an unsupported factual claim.
• Adding findings without revising the recommendation they contradict.
• Treating no search result as proof that a claim is false.
• Calling the answer complete because the iteration or spend budget ended.

Where the exam guide differs

What to remember

Evaluate against explicit criteria, diagnose the missing work, and delegate only what can repair it. Re-synthesize from the updated evidence and check the entire answer again. Stop when material findings are sufficient, or disclose the unresolved boundary without presenting a budget stop as success.

Knowledge check

References

• Anthropic, How we built our multi-agent research system
• Anthropic, Building effective agents
• Anthropic, Subagents in the SDK
• Anthropic, Create custom subagents
• Anthropic, Claude Certified Architect – Foundations Exam Guide