Validate totals against their source evidence

Learning objective

Validate source-grounded semantics with application validators (e.g., Pydantic), including calculated versus stated totals and detected conflicts.

Core idea

A record can satisfy every type requirement and still misrepresent its source. An integer in stated_total might contain the wrong amount; a valid date in invoice_date might be the due date. Your application needs checks that compare relationships and source evidence, beyond checking whether the record parses.

Preserve the distinction between an extraction defect and a document defect. If Claude misreads a printed total, correct the extraction. If the printed total disagrees with correctly extracted lines, retain both figures and flag the source inconsistency. Making the figures agree by changing the printed value would destroy the evidence you need to review.

See 4.3.6 for the schema-versus-semantics distinction. Here you turn that distinction into validators, calculated-versus-stated fields, and conflict handling.

How it works

Assign each check to the right layer

For Claude tool extraction, input_schema describes the record and strict: true constrains tool inputs to the supported schema. Without strict mode, Anthropic documents that inputs can contain incompatible types or omit required fields. For native JSON responses, output_config.format supplies the structured output contract. These mechanisms govern structure, not whether an amount accurately represents an invoice.

Pydantic adds application validation. In Python, a BaseModel declares the fields, and @model_validator(mode="after") checks relationships after the model’s fields have been validated. An after validator returns the validated instance; raising ValueError reports a failed rule through validation. Use a rule whose meaning is specific enough to produce useful feedback.

The following layers catch different failures:

Preserve both totals

Define stated_total as the amount printed in the supplied document. Define calculated_total as the result of the specified calculation over extracted components. Give each field a precise description in the extraction schema. These are fields you design, not built-in Anthropic response fields.

See 4.3.6 for choosing the calculation and comparing like quantities; the simplified validator here sums all charges as integer cents in one currency, with no tax or discount.

This short Python example checks the model’s arithmetic claim against its extracted components:

from pydantic import BaseModel, model_validator


class Invoice(BaseModel):
    charges: list[int]
    stated_total: int
    calculated_total: int
    conflict_detected: bool

    @model_validator(mode="after")
    def verify_calculation(self):
        expected = sum(self.charges)
        if self.calculated_total != expected:
            raise ValueError(
                f"calculated_total={self.calculated_total}; "
                f"sum(charges)={expected}"
            )
        return self

Instantiate this model with the extracted values. The example deliberately checks only arithmetic consistency. It cannot establish that the charges were read correctly, that none were omitted, or that the stated total matches the printed text. Those require comparison with the source.

After this check, compare the recomputed amount with stated_total. A difference is evidence to investigate, not a license to replace stated_total. Keep the original amount available for that investigation.

Verify conflicts rather than trusting a flag

Ask the extractor to set conflict_detected when the source contains inconsistent values and to identify the conflicting passages in an application-defined explanation field. Anthropic recommends grounding document answers in direct quotes and checking claims against their supporting text. Apply that principle to a conflict claim: preserve the locations and values that supposedly disagree.

A true flag does not prove the conflict is real. A false flag does not prove consistency. Your validator can still detect an arithmetic difference when the flag is false. Conversely, a model might flag two amounts that serve different purposes, such as subtotal and grand total, even though the source is coherent.

Separate the validator’s result from the model’s flag. The application decides whether to accept the record, request a correction, or retain it for review. See 4.4.2 for correction requests and 4.4.4 for how the flag contributes to failure analysis.

Key distinctions and trade-offs

Schema validity versus source fidelity: Strict output makes a supported structural contract enforceable. It does not turn a correctly typed number into a verified transcription.

Independent arithmetic versus repeated model arithmetic: Comparing two model-produced numbers exposes differences, but both can be wrong together. Recomputing from the extracted components gives you a separate arithmetic check; inspecting the source gives you the transcription check.

Extraction mismatch versus source conflict: A mismatch identifies disagreement. Its cause determines the next action. Correct a misread value; preserve a verified conflict in the document.

Application checks also need domain assumptions. A date-order rule can identify suspicious field placement, but it does not prove which date was copied incorrectly. Treat the rule as a pointer to the relevant source labels, rather than swapping the dates automatically.

Architecture / decision scenario

A structured data extraction service processes an invoice with charges of 180.00, 275.00, and 45.00 and a printed total of 500.00. Claude faithfully extracts charges: [18000, 27500, 4500] and stated_total: 50000 in the service’s cents convention, but returns calculated_total: 55000. The record passes its structural contract; the model’s arithmetic is wrong.

The application’s Pydantic @model_validator(mode="after") recomputes 50000 and rejects the record with calculated_total=55000; sum(charges)=50000. The error targets the calculated field without changing the faithfully transcribed stated amount. The correction flow from 4.4.2 receives the source, failed record, and exact error, then the application validates the replacement again.

If a recomputed total differs from the printed amount, use the extraction-defect versus source-defect decision in 4.3.6. That fork depends on source comparison, rather than assuming that either number must be replaced.

A second invoice returns conflict_detected: true with two quoted passages: “Payment due 20 November” in its header and “Payment due 27 November” on its payment slip. The service checks both quotes against the supplied document and confirms that they refer to the same payment obligation. It retains the conflicting dates and routes the record to review. If either quote is unsupported, the flag is an extraction claim to correct, not a verified source conflict.

Common exam traps

• Treating a well-typed amount as a verified source value.
• Comparing a subtotal with a grand total without specifying the calculation.
• Trusting calculated_total without recomputing from the components.
• Replacing a source’s printed total to make a validation check pass.
• Skipping validation because conflict_detected is true.
• Automatically swapping dates because one ordering rule failed.

Where the exam guide differs

What to remember

Keep the printed value, expose the calculated value, and recompute independently. Pydantic model validators enforce relationships you define; they do not verify a document by themselves. Investigate mismatches, preserve genuine source conflicts, and verify the evidence behind conflict_detected.

Knowledge check

References

• Anthropic, Strict tool use
• Anthropic, Structured outputs
• Pydantic, Validators
• Anthropic, Reduce hallucinations
• Anthropic, Claude Certified Architect – Foundations Exam Guide