Skip to lesson

AnthropicClaude Certified Architect – Foundations2026-105.5 Design human review workflows and confidence calibration

Send unresolved evidence to targeted review

Completed by the knowledge check

Learning objective

Prioritize human review for low-confidence fields, ambiguous evidence, and contradictory documents.

Core idea

For an architect allocating limited extraction reviewers, confidence is one routing signal. A calibrated low score identifies a field worth checking. Ambiguous evidence or contradictory documents create a different problem: the source may not establish a single answer, even when Claude reads each passage confidently.

The exam guide requires review for all three conditions. Design the application to preserve the proposed value and its evidence, identify why review is needed, and send the smallest useful review scope to a person. See 5.5.2 for threshold calibration and 5.5.3 for random audits of apparently sound results that bypass this queue.

How it works

Separate the three review reasons

A low-confidence field is one whose score falls below its validated field threshold. The source might contain a clear answer that the extractor failed to recover reliably. A reviewer can often resolve that case by inspecting the field’s source passage.

Ambiguous evidence supports more than one interpretation or omits a necessary convention. The text 04/05/2026 can remain unresolved without a documented date convention. A high confidence score for reading those characters does not settle their interpretation. See 4.3.4 for representable missing and uncertain values and 4.3.5 for normalization without invented evidence.

Contradictory evidence makes incompatible assertions about the same field. An invoice header and its payment slip might state different due dates, or an invoice and its amendment might disagree about the total. First verify that they refer to the same obligation and that the discrepancy is real. See 4.4.1 for checking reported conflicts against source values. Once verified, a confidence score cannot choose which assertion has authority.

These reasons can overlap. A field may have low confidence because two sources conflict. Preserve both reasons rather than collapsing every case into “low score.” The distinction tells the reviewer whether to inspect legibility, clarify a convention, or resolve source authority.

Make routing explicit in the application contract

Request the evidence needed for review alongside the extracted values. Anthropic’s hallucination guidance recommends admitting uncertainty, grounding factual claims in direct quotes, and checking claims against sources. Apply that guidance in the extraction prompt: place supporting and competing source quotes in an application-defined evidence property beside each value in the extraction schema, and avoid guessing when the documents cannot settle the value. The API’s Citations feature, enabled with "citations": {"enabled": true} on a document block and returning cited_text, cannot be combined with output_config.format: that combination returns a 400 error, so this structured extraction carries evidence as schema fields. See 5.6.1 for preserving claim-to-source mappings and locations.

In this illustrative application record, confidence is the raw field score, ambiguity_detected marks unresolved interpretation, and conflict_detected marks incompatible evidence. A review_reasons list and review_status record what the application decides. All five names are application-defined; they are not Claude response fields or a built-in human-review API. The diagnostic use of conflict_detected is introduced in 4.4.4.

The application compares confidence with the threshold stored for the field and validated segment. It checks ambiguity and conflict evidence independently of that comparison. Its decision rule is:

if segment is not validated for acceptance: require review
else if evidence is ambiguous or conflicting: require review
else if field confidence is invalid or below its threshold: require review
else: accept, subject to the ongoing audit policy

The rule belongs to application control flow after extraction. A prompt can request an uncertainty flag, but review enforcement should not depend on Claude deciding whether to call a reviewer. Validators can also identify contradictions the model fails to flag, such as incompatible printed totals. A false flag should be checked against the source rather than accepted as an authoritative diagnosis.

Match the review scope to the unresolved evidence

For a single doubtful field, show the proposed value and source passage together. Include the threshold comparison and reason for review. The reviewer can confirm or correct the field without rereading every reliable value in the record.

For conflicting documents, include both documents, their relevant passages, and any known relationship between them. A corrected invoice can supersede an earlier one only when the application has evidence or a defined business rule establishing that relationship. Do not silently select the last uploaded document as authoritative.

A reviewer outcome should distinguish confirmed, corrected, and unresolved cases. These are application statuses, not Anthropic enums. Record the accepted value and the supporting evidence when resolved; retain an unresolved state when a supplier must clarify. A reviewer must be able to decline to choose, because reading the same ambiguous source again does not necessarily create the missing fact.

Key distinctions and trade-offs

Field-level review saves attention when the doubt is local. Document-level review is appropriate when the uncertainty affects related values or source authority. If an amendment changes both total and tax, reviewing only one flagged total can leave a contradictory record. Expand the scope to the values whose correctness depends on the decision.

Review priority and acceptance eligibility are separate. Rank unresolved cases by business consequence and dependencies, but do not auto-accept a conflicting value merely because the queue is long. A disputed total blocking payment may deserve attention before an optional purchase-order reference; both retain their required-review status until resolved.

Model confidence describes an extraction signal. Source authority decides which evidence controls the business record. Human judgment resolves the latter when the documents and policy cannot. Repeating the extraction with a demand for greater confidence changes neither source authority nor missing evidence.

Reviewer corrections provide labeled outcomes for later evaluation, but the review queue is biased toward difficult cases. Use it to repair those cases and retain the separate accepted-set audit from 5.5.3. Otherwise, the quality report measures only the work the router already suspected.

Architecture / decision scenario

A distributor has validated field-level thresholds for typed invoices. One record contains a clear invoice number, a total below its confidence cutoff, and a due date above its cutoff. The due date appears twice with incompatible values. Another invoice uses a date whose month/day convention is unspecified.

The application accepts the clear invoice number and sends the low-confidence total to field review. It sends the conflicting due date to review despite the passing score, with both passages attached. The separate ambiguous date also goes to review; its high score does not establish a date convention.

The reviewers prioritize the disputed payment amount and deadline because those values block the payment run. They confirm the total from the source. For the due date, they request supplier clarification because neither passage establishes authority. The record remains unresolved instead of receiving whichever date appeared first.

Once clarification establishes the correct date, the reviewer records the resolved value and its evidence. The application then permits the dependent payment workflow to continue. It retains the low-score correction and the conflict outcome for quality analysis, while independently auditing accepted fields. Limited reviewer capacity changes the order and scope of attention, not the requirement to resolve contradictory evidence.

Common exam traps

  • Letting a high confidence score override ambiguous or contradictory source evidence.
  • Assigning one score to a whole record and reviewing every field unnecessarily.
  • Treating conflict_detected as a built-in Claude field or a verified fact without checking its evidence.
  • Blanking a low-confidence value instead of preserving a candidate and its source for review.
  • Forcing repeated extraction when only an external clarification can settle the answer.
  • Prioritizing the queue by accepting unresolved low-priority cases without evidence.

What to remember

Use calibrated confidence for uncertain extractions and independent evidence checks for ambiguity and contradiction. Route locally doubtful fields to targeted review and source-authority disputes with all relevant documents. Preserve unresolved outcomes until evidence supports a decision; reviewer scarcity governs attention, not factual correctness.

Knowledge check

References