Separate findings by their evidential support

Learning objective

Separate well-supported, disputed, and under-researched findings while retaining methodological qualifiers.

Core idea

A report can contain strong evidence, real disagreement, and unanswered questions at the same time. Presenting them as one confident narrative hides which conclusions the reader can use and which need further investigation. A disclaimer at the end cannot repair individual assertions that overstate their evidence.

For a research-system architect, the task is to make evidential status survive synthesis. Require distinct report sections and keep each finding’s scope and methodology attached. This lesson concerns the status of conclusions, rather than model confidence calibration or the mechanics of a failed search.

How it works

Define the report categories

Use an application-defined evidence_status with a documented interpretation. For this report design, the values well_supported, disputed, and under_researched map to three visible sections. These labels are your contract, not Anthropic API enum values or a native automatic evidence classifier.

A well-supported finding has evidence sufficient for the claim as phrased. Agreement among independent credible sources can strengthen that support, but several pages repeating one original report are not several independent measurements. A narrow, directly supported conclusion can be useful even when broader generalization is unjustified.

A disputed finding has credible competing evidence that remains unresolved after the coordinator’s treatment in 5.6.3. Keep the competing claims with attribution. An unsupported rumor does not automatically deserve equal standing with a documented measurement; the record must preserve the basis for treating evidence as credible.

An under-researched finding lacks adequate coverage for the requested conclusion. Name what remains unknown and what evidence is missing. See 5.3.4 for carrying partial coverage and access outcomes. Here the important distinction is that a gap does not establish either agreement or disagreement about the unanswered question.

Keep qualifications inside the finding

Preserve the source’s characterization in an application-defined qualifiers field and its method in method. Useful qualifiers include preliminary estimates, forecasts, narrow samples, self-reported outcomes, and measurements limited to a particular test configuration. Extract them with the supporting passage, not as an optional afterthought.

A finding can be well supported within a limited scope. A laboratory benchmark might establish performance on one workload without establishing fleet-wide performance. Classify the narrow claim according to its evidence and retain the limitation; do not turn every qualified measurement into an unusable uncertainty bucket.

Similarly, an optimistic scenario is not a forecast certainty. If a source says a result depends on a favorable adoption assumption, the report must carry that assumption alongside the result. Copying the value while removing its condition changes the claim, even if the source URL survives.

Anthropic’s hallucination guidance recommends extracting direct quotes, verifying claims with supporting passages, allowing uncertainty, and restricting answers to provided material. Use those instructions to protect qualifiers during synthesis. Ask the agent to check that each conclusion preserves the population, conditions, and evidential status expressed by its sources.

Require visible sections at synthesis time

Put the category definitions and section requirements in the synthesis specialist’s AgentDefinition.prompt. Include the coordinator’s classification decisions with the evidence it receives. Anthropic’s multi-agent research engineering post recommends giving subagents objectives, output formats, source guidance, and clear boundaries; evidential status belongs in that output contract.

Require the report sections “Well-supported findings,” “Disputed findings,” and “Research gaps.” Within each, retain claim and evidence identifiers so the status remains traceable. If a section has no findings, state that rather than filling it with speculative content to make the report look balanced.

Then verify the rendered statements against the underlying records. A section heading is not sufficient if a disputed estimate appears elsewhere as an unqualified fact. A strong executive summary can state the supported conclusion and immediately identify the unresolved dependency that limits it.

When follow-up research changes a classification, update the finding and the report together. A resolved methodological difference can move a claim out of the disputed category. A discovered sampling limitation can narrow a well-supported statement. Keep the reason for the change available to the coordinator rather than silently replacing the record.

Key distinctions and trade-offs

Evidence status versus confidence score: The categories describe support and coverage in the available material. They are not percentages produced by the model and do not certify that a claim is true.

Qualification versus dismissal: A limitation controls the scope of a conclusion. It does not automatically invalidate the observation. Preserve useful narrow findings while preventing broader unsupported claims.

Disputed versus under-researched: Dispute needs competing credible evidence. Under-research needs an explicit coverage gap. Combining the two hides whether the next step should reconcile existing records or obtain missing evidence.

Separate sections can interrupt narrative flow, but they let the reader distinguish usable findings from unresolved decisions. Keep the summary concise while placing qualifications where they affect interpretation. More confident wording is not a substitute for stronger evidence.

Architecture / decision scenario

A coordinator investigates a fictional database product for an engineering evaluation. Three specialists return a published benchmark, two incompatible estimates of production adoption, and an incomplete search for recovery-test results.

The benchmark reports lower latency for a specific synthetic workload on a named test configuration. The coordinator classifies the scoped performance statement as well supported and retains the workload and hardware qualifiers. Synthesis does not expand it into “the product is faster for every production application.”

The adoption estimates remain incompatible after the available methodology review. They appear in the disputed section with both sources and the coordinator’s unresolved treatment. The recovery specialist found no adequate test evidence within its coverage; synthesis places that question under research gaps and explains that recovery performance remains unestablished.

The report’s summary says the benchmark supports further evaluation for that workload, adoption remains disputed, and recovery requires targeted testing evidence. The final claim review checks supporting passages and removes an unsupported assertion that recovery is unreliable. The report preserves all three statuses without weakening the benchmark result or converting missing research into a negative finding.

Common exam traps

• Replacing explicit evidence categories with a single confidence percentage.
• Treating repeated reporting of one study as independent corroboration.
• Removing “preliminary,” “forecast,” or a test condition during compression.
• Calling an unanswered question a disputed finding.
• Treating missing recovery evidence as proof of poor recovery.
• Adding a global disclaimer while leaving individual claims overstated.

What to remember

Separate supported findings, credible unresolved disputes, and research gaps. Keep methodological qualifiers with the specific claims they limit. A trustworthy synthesis can contain a useful narrow conclusion and an unresolved broader question without collapsing either into a uniformly confident story.

Knowledge check

References

• Anthropic, Reduce hallucinations
• Anthropic, Subagents in the SDK
• Anthropic, How we built our multi-agent research system