Define which defects deserve a finding

Learning objective

Replace vague review prompts with explicit categorical inclusion and exclusion criteria for reportable defects.

Core idea

For an architect designing a Claude code-review workflow, the first decision is what counts as a defect. A prompt such as “review this change carefully” gives a goal but leaves the reporting boundary undefined. A correct observation about formatting can become a false positive when formatting is outside the review’s purpose.

Define a reportable finding as a category-specific condition supported by code evidence. State both the inclusion rule and its exclusions. Claude should not have to infer whether your team wants production bugs, naming suggestions, comment rewrites, or all three.

Anthropic’s prompting guidance recommends explicit instructions and output constraints. The Claude Code best-practices section “Add an adversarial review step” warns that a subagent prompted to find gaps can produce findings even when the work is sound. The remedy is to specify correctness or stated requirements as the reporting boundary. This lesson applies that guidance to a review contract; see 4.1.2 for confidence filtering and 4.1.3 for severity after a finding qualifies.

How it works

Replace each goal with a test

A categorical rule names the issue type, the evidence that admits it, and the cases that do not qualify. “Check that comments are accurate” fails because accuracy can mean completeness, clarity, freshness, or agreement with behavior. Each interpretation admits a different set of findings.

Replace it with a contradiction test: report a comment only when it makes a behavioral claim that the implementation contradicts. Require the reviewer to identify the claim and the conflicting code path. A terse comment is not a contradiction. A TODO without a behavioral claim is not a contradiction. A comment promising three attempts above code that performs one attempt is a candidate, provided no caller or helper supplies the other attempts.

That last qualifier matters. A local snippet may look inconsistent while the complete execution path satisfies the comment. Give Claude the relevant surrounding implementation and require it to account for delegating helpers before declaring a mismatch. The rule should define the evidence needed, rather than turn an incomplete view into a confident accusation.

Make category actions explicit

Use a category-action table in the review instructions. The following is an illustrative team policy, not a built-in Anthropic taxonomy:

Exclusion rules describe what does not qualify. They must not excuse real defects merely because those defects resemble a familiar pattern. For example, a legacy return-code convention can be excluded as a stylistic complaint while a caller ignoring a failure code still qualifies as a behavioral bug.

Carry the rule into the finding

Ask the reviewer to provide the category, location, violated criterion, triggering input or path, and observed effect for each finding. Requiring the violated criterion makes it possible to check whether the reviewer applied your definition or substituted its own.

A compact prompt clause can express the decision:

Report a finding only if it satisfies an inclusion rule and survives
that category's exclusions. Name the rule, code location, triggering
condition, and incorrect effect. Return no findings when none qualify.
Do not invent a defect to populate a category.

Put rules in the review’s instruction file

For managed Claude Code Code Review, put inclusion and exclusion rules in REVIEW.md at the repository root. Finding and verification agents receive its contents as review instructions; ranking and reporting agents consult it before settling severity and writing findings.

The documented patterns give the category table a concrete home: Skip rules suppress findings for named categories, paths, or branch patterns, including checks CI already enforces. Repo-specific checks define what to check on every pull request. Verification bar requires evidence, such as a file:line source citation for behavioral claims rather than an inference from naming. The documentation’s example uses ## Do not report for exclusions and ## Always check for mandatory checks; these are freeform Markdown headings, not configuration keys.

Code Review is in research preview for Team and Enterprise subscriptions. It is unavailable to organizations with Zero Data Retention enabled or the HIPAA configuration applied. The local /code-review command follows CLAUDE.md but does not read REVIEW.md. For a custom claude -p or subagent reviewer, supply the category table and decision clause in its own task prompt. Verify both qualifying findings and empty-report cases.

Key distinctions and trade-offs

Observation versus reportable defect: An observation can be true and still fall outside your reporting contract. “This name is abbreviated” does not establish a wrong result or a violated requirement.

Category versus severity: Category decides which rule applies. Severity ranks the effect of a qualifying issue. A critical-sounding category name cannot replace evidence that a defect exists.

Exclusion versus blanket immunity: Excluding naming preferences preserves review focus. Excluding every issue in code that follows a local pattern can conceal actual bugs. Write the exclusion around the absence of a qualifying effect.

Narrow rules reduce irrelevant findings but can omit issue types you forgot to include. Review the category table against the system’s actual risks. Do not expand it with vague catch-all instructions such as “also mention anything concerning,” which restore the ambiguity the table removed.

Architecture / decision scenario

A developer-productivity team runs Claude Code on pull-request changes. The review reports valid authorization bugs but also complains about abbreviated names and old TODOs. The team wants security, behavioral bugs, and genuine comment contradictions; its linter owns formatting.

For managed Code Review, the architect puts the category-action table in REVIEW.md; for a custom reviewer, the architect supplies it in the review prompt. For comment findings, the prompt requires an exact behavioral claim and the conflicting implementation. For behavioral bugs, it requires a realistic triggering condition and wrong effect. It explicitly excludes naming preferences, harmless TODOs, and consistent local conventions without a functional defect.

In a test change, a refund caller ignores a failure return code, a nearby helper follows the established return-code convention, and a comment promises retries that no helper performs. The caller qualifies as a behavioral bug. The helper’s convention does not qualify merely because exceptions would be another design choice. The retry claim qualifies as a comment contradiction. Claude can also return an empty result for a change containing only naming differences.

The team evaluates those category decisions before tuning severity or confidence. The defect definition is the first repair because it determines which findings should exist at all.

Common exam traps

• Replacing “check accuracy” with “be very accurate” without defining a test.
• Treating every correct observation as a reportable defect.
• Defining inclusion rules while leaving style and local-pattern exclusions implicit.
• Exempting a familiar pattern even when a concrete bug satisfies the inclusion rule.
• Requiring at least one finding per category, which makes a clean review impossible.

What to remember

Define reportable defects by category, evidence, and effect. Give each category explicit exclusions and permit an empty report. For comments, test claimed behavior against implemented behavior; do not substitute judgments about tone, age, or completeness.

Knowledge check

References

• Anthropic, Prompting best practices
• Anthropic, Best practices for Claude Code
• Anthropic, Code Review