Stop retries when source evidence is absent

Learning objective

Stop futile retries when required evidence is absent from the supplied document or needs unavailable external material.

Core idea

A correction request can redirect attention to evidence that Claude overlooked. It cannot supply a fact that is absent from the request. Before retrying an incomplete extraction, decide whether a careful reader with the same supplied material could recover the missing value.

If the invoice contains the payment terms in a footer, a targeted reread can help. If it says “payment terms are defined in the supply agreement” and the agreement is unavailable, an identical-source retry cannot establish those terms. Increasing the correction count adds attempts without changing the evidence.

See 4.4.2 for constructing a useful correction request. This lesson owns the decision to leave that path when evidence is missing, including how to preserve an incomplete result without turning absence into an invented value.

How it works

Classify the evidence problem

Inspect the supplied source and the failed extraction before choosing a remedy. Anthropic recommends grounding document responses in direct quotes and explicitly permitting uncertainty. Ask for the supporting passage for a claimed value; if no passage supports it, do not treat a confident answer as evidence.

The following cases need different responses:

A wrong field placement is another correctable case: the source contains the value, but the extraction assigns it to the wrong label. A format mismatch is correctable only when normalization does not require guessing. A date such as 03/04/26 without a known convention can remain ambiguous even though its characters are visible.

Distinguish absence from an incomplete search

An empty field is a symptom, not a diagnosis. Claude might have overlooked a table cell, or the supplied document might be incomplete. Ask for the evidence relevant to that field, including any explicit reference to an attachment. Compare that explanation with the material the application actually supplied.

For an invoice that references an agreement, the quoted reference proves that the invoice points elsewhere. It does not prove what the agreement says. Keep those two conclusions separate. The service can identify an evidence dependency without filling the dependent value.

If the value is visible in the supplied source, return to the correction path with its location and failed field. If the source is unreadable or omits pages, seek a usable copy rather than treating repeated extraction attempts as a substitute for the missing content. The deciding change is better evidence, not a stronger demand for an answer.

Give missingness a valid representation

Anthropic’s hallucination guidance recommends giving Claude permission to say it lacks enough information and instructing it to use only provided documents. Put both instructions in the extraction prompt. State that external conventions, supplier habits, and general knowledge do not count as evidence for a document-specific field.

The output contract must support that instruction. Use the missing-value design in 4.3.4, such as a required field that accepts null. A required key means the key must be present; it does not mean the source necessarily contains a non-null value. Pair missing values with an application-defined explanation or status so downstream consumers understand why the record is incomplete.

For example, an extractor can return payment_terms: null and explain that the supplied invoice refers to an unavailable agreement. The application routes that result to its incomplete-record queue. It does not send a validation error that says “payment_terms must contain a plausible term,” because that instruction rewards fabrication.

If a downstream payment workflow requires complete terms, keep the record out of that workflow. The extraction result can accurately represent missing evidence while still failing the business requirement for a payable invoice. Structural acceptance and permission to proceed are separate decisions.

Change the evidence before extracting again

When an authorized workflow obtains the referenced agreement, include it in the next request alongside the invoice. The Messages API is stateless, so the request must contain the source context you want Claude to use. A message saying “the agreement is now available” does not itself provide its contents.

Treat this as an extraction with an expanded evidence set. It can recover a value that the invoice alone could not support. Preserve which document supports the terms, so the result does not falsely attribute them to the invoice.

If the external material remains unavailable, stop and retain the reason. Human review can decide how to obtain the source or whether the incomplete record is usable for another purpose. It should not be asked to approve a model’s unsupported guess as though it were a transcription.

A same-source retry helps only overlooked evidence, while absent evidence needs a changed evidence set before Claude extracts again.

Key distinctions and trade-offs

Absent versus overlooked: Both can produce null. Only overlooked evidence gives a same-source correction request something to recover.

Missing value versus invalid representation: A missing value can be an accurate extraction outcome. A malformed representation is an output defect. A stricter shape cannot create missing content.

Correction request versus evidence acquisition: Correction rereads supplied material. Acquisition changes what is supplied. Once a required fact exists only in another document, acquiring that document is the relevant action.

Allowing missingness reduces apparent completeness, but preserves source fidelity. Requiring a non-null value for every field can make a record look complete while hiding unsupported assumptions. Measure success by accurate evidence handling, including correct incomplete outcomes, rather than the percentage of filled cells alone.

Architecture / decision scenario

A structured data extraction service handles three invoices. The first includes “Net 45” in a footer, but the extraction returns payment_terms: null. The application verifies that passage and sends a targeted correction request. The same-source retry is justified because the answer is present.

The second says “Terms per attached agreement,” but no agreement is supplied. The service retains a null value and the quoted dependency, stops correction requests for that field, and asks its document workflow for the agreement. If the workflow cannot provide it, the invoice stays incomplete. Neither more examples nor a larger retry limit establishes the missing terms.

The third contains two conflicting payment clauses. It is not an absent-value case: both passages are present, but no supplied authority resolves their disagreement. The service records the conflict for review rather than prompting Claude to choose the more familiar term.

If the second invoice’s agreement becomes available, the service sends both documents in an expanded evidence request. It validates the extracted terms against the agreement and preserves that attribution. This is progress because the input evidence changed, not because another identical attempt eventually produced a value.

Common exam traps

• Treating every empty required field as a correctable extraction error.
• Raising retry counts when the answer exists only in an unavailable attachment.
• Filling supplier terms from general knowledge or examples of other invoices.
• Treating a model’s confident non-null answer as proof that evidence exists.
• Rejecting an accurate null result because the business workflow needs completeness.
• Calling a request with additional source material an identical-source retry.

What to remember

Retry when supplied evidence can correct the failed record. Stop when the fact is absent, unreadable, or dependent on unavailable material. Preserve missingness, state the evidence gap, and extract again only when the necessary source context becomes available.

Knowledge check

References

• Anthropic, Reduce hallucinations
• Anthropic, Structured outputs
• Anthropic, Using the Messages API