Diagnose failures with pattern and conflict fields

Learning objective

Record detected_pattern in review findings and conflict_detected in extractions to diagnose different failure classes.

Core idea

A code-review finding needs to identify the construct that triggered it. An extraction needs to identify whether its source contains inconsistent information. detected_pattern and conflict_detected answer those different questions; neither field should stand in for the other.

Task statement 4.4 of the exam guide names both fields: detected_pattern for feedback-loop analysis and conflict_detected in self-correction validation flows. You define them in your application’s output schema and instructions. Their diagnostic value comes from clear meanings, evidence, and downstream analysis, rather than from the names alone.

See 4.4.1 for validating source conflicts. Here you use reviewed outcomes to distinguish noisy review triggers, extraction mistakes, and genuine document inconsistencies. The decision is which part of the system needs correction, rather than whether every reported issue should be suppressed.

How it works

Record the trigger separately from severity

Define detected_pattern as the code construct that prompted a review finding. For example, broad_exception_catch describes a catch structure, while sql_string_concat describes a query-building structure. These are example application labels, not Anthropic enums. Neither label proves the code is defective.

Keep the location, issue, severity, and supporting evidence in their own fields. See 4.2.3 for an actionable finding’s presentation. A severity such as “high” cannot identify which construct generated the finding, and a vague category such as “quality” does not let you distinguish unrelated triggers.

A short application-defined finding record might look like this:

{
  "file": "billing/export.py",
  "line": 88,
  "severity": "minor",
  "issue": "Failure is discarded before the caller receives a result.",
  "detected_pattern": "broad_exception_catch"
}

The label records the trigger; the issue states the claimed defect. A broad catch that intentionally converts errors into a documented result could share that label without deserving a finding. The distinction lets you discover whether the reviewer confuses a syntactic pattern with a harmful behavior.

For an API-built reviewer, use input_schema with strict: true for findings emitted through a tool, or output_config.format for native JSON findings. A string enum can constrain the intended pattern vocabulary to your defined categories. Anthropic’s Structured outputs page documents a capitalization caveat for string enum values in both JSON outputs and strict tool use: compare them case-insensitively and avoid labels that differ only by capitalization. Structural enforcement does not establish that the selected label matches the code.

For a Claude Code reviewer, use claude -p --output-format json --json-schema 'SCHEMA' "Review the supplied diff". Replace SCHEMA with the JSON Schema defining your findings, including detected_pattern. The --json-schema flag works in print mode only; the response envelope holds the schema-conforming result in structured_output. See 3.6.2 for consuming that result. The API casing caveat described here concerns the two interfaces named by the Structured outputs page; the headless documentation does not establish whether it applies to this CLI flag.

Join findings to reviewed outcomes

Store the finding’s label when it is emitted, then associate the developer’s disposition and dismissal reason with that same finding. A developer dismissing “this exception is swallowed” might explain that the wrapper records the error and returns a failure result. That reason is more informative than an undifferentiated dismissal count.

Group reviewed findings by detected_pattern. For each group, count dismissals and the reviewed findings from which they came. A group with eight dismissals among ten reviewed findings differs from one with eight dismissals among one hundred. Keep the denominator visible so raw counts do not masquerade as comparable rates.

Treat dismissal rates as signals for inspection, not automatic proof of false positives. A developer can dismiss a real defect because it is accepted risk, belongs to another change, or lacks enough context. Check the dismissal reasons and relevant code before labeling a pattern unreliable.

Anthropic’s evaluation guidance recommends specific, measurable criteria and task-specific cases, including ambiguous edge cases. Apply that guidance to the noisy pattern: build examples of valid uses and real defects, revise the criteria, and evaluate the revised reviewer against both. See 4.1.4 for temporarily suppressing a noisy category and 4.2.2 for contrast pairs. The pattern label tells you which evidence set to inspect; those lessons own the repair techniques.

Preserve source conflicts as a separate signal

In extraction records, define conflict_detected as a boolean indicating a reported disagreement within the supplied source. Include an explanation identifying the conflicting fields, values, and passages. Anthropic’s grounding guidance recommends direct quotes and verifying claims against supporting text; a conflict claim needs that evidence too.

See 4.4.1 for the validators and source checks that distinguish erroneous extracted totals from verified source conflicts while preserving stated_total and calculated_total.

Record the verified outcome separately from the model’s boolean, so feedback can reveal false conflict claims as well as missed conflicts.

The signals support different analyses:

Do not collapse them into a single failure count. An extraction system that correctly flags a contradictory invoice has done useful work; a reviewer that repeatedly flags harmless wrappers needs different changes.

Key distinctions and trade-offs

Pattern versus verdict: A construct can appear in both correct and defective code. Its label explains the trigger, while the finding’s evidence explains the alleged defect.

Dismissal versus false positive: A dismissal is observed feedback. A false positive is a judgment established by inspecting the claim and code. Keep the reason that connects them.

Reported conflict versus verified conflict: conflict_detected records what the extraction says. Application and source checks establish whether that statement is justified.

A stable vocabulary makes aggregation useful, but overly broad labels hide distinctions. Conversely, a separate label for every finding makes groups too small to analyze. Choose labels around repeatable code constructs, then use the issue and dismissal reason for context. Keep the field’s meaning consistent across evaluation runs so a changed vocabulary does not look like a quality improvement.

Architecture / decision scenario

A Claude Code CI reviewer runs in print mode with --output-format json and --json-schema, then reads findings containing detected_pattern from structured_output. Developers dismiss eight of ten reviewed broad_exception_catch findings but only one of twenty reviewed sql_string_concat findings. The team reads the first group’s reasons and discovers that several wrappers intentionally return failure objects instead of throwing.

The team revises the broad-catch criteria and evaluates the change on harmless wrappers and genuine swallowed failures. It preserves the second category’s criteria. A global instruction to “report fewer issues” would obscure which trigger was noisy and could remove useful detections elsewhere.

The same team operates an invoice extractor. Its conflict_detected rate rises, but checked invoices genuinely contain conflicting totals. Those successful detections should not be dismissed as model noise. Another record has conflict_detected: true because the extractor misread a digit; source inspection identifies an extraction defect, which enters the correction flow instead.

The team therefore reports reviewed outcomes separately: finding dispositions by trigger, verified source conflicts, and corrected extraction errors. Each signal directs a different intervention, and the evaluation checks that the intervention improves the intended behavior.

Common exam traps

• Recording severity without the construct that triggered the finding.
• Treating every broad catch as a defect because of its pattern label.
• Calling all developer dismissals confirmed false positives.
• Comparing raw dismissal counts without reviewed denominators.
• Using conflict_detected as a code-review category or detected_pattern as a source-conflict flag.
• Counting accurate source-conflict reports as failed extractions.

What to remember

Record the code trigger in detected_pattern and the reported source inconsistency in conflict_detected. Join each signal to evidence-backed outcomes. Analyze reviewed findings by pattern, verify conflict claims, and target changes at the failure class the evidence actually supports.

Knowledge check

References

• Anthropic, Run Claude Code programmatically
• Anthropic, CLI reference
• Anthropic, Structured outputs
• Anthropic, Define success criteria and build evaluations
• Anthropic, Reduce hallucinations
• Anthropic, Claude Certified Architect – Foundations Exam Guide