AnthropicClaude Certified Architect – Foundations2026-104.1 Design prompts with explicit criteria to improve precision and reduce false positives
Anchor severity labels in observable effects
Completed by the knowledge check
Learning objective
Define severity levels using concrete effects and code examples for consistent classification.
Core idea
For an architect using Claude to classify review findings, a severity label needs a definition that another reviewer can apply. “Critical,” “high,” “medium,” and “low” do not establish shared boundaries on their own. Without effects and examples, the same defect can receive different labels in similar reviews.
Define each level by the consequence of a qualifying defect, then give a representative code example with the context that makes it fit. Ask Claude to report the matching criterion and demonstrated effect alongside the label. The rubric becomes a testable classification contract instead of a collection of adjectives.
Anthropic’s prompting documentation recommends explicit constraints and relevant, diverse examples to improve output consistency. Its evaluation guidance supports well-defined qualitative scales and expert rubrics. The four-level rubric here is an illustrative custom-harness policy; managed Code Review has its own documented markers and calibration mechanism.
How it works
Classify only an eligible finding
Apply the defect inclusion rules from 4.1.1 before assigning severity. A naming preference does not become a valid defect because Claude labels it low. A verified authorization failure does not stop qualifying because it resembles an established local pattern.
This order prevents severity from becoming a second, ambiguous defect detector. Category identifies the applicable inclusion rule. Evidence establishes that the rule is met. Severity then describes the effect. Confidence, discussed in 4.1.2, remains a separate judgment about how well the evidence supports the finding.
Calibrate the managed review’s markers
Managed Code Review tags findings as 🔴 Important (a bug to fix before merging), 🟡 Nit (a minor issue worth fixing but not blocking), or 🟣 Pre-existing (a bug not introduced by the pull request). Its REVIEW.md Severity pattern lets you redefine Important for your repository and state which finding classes are Important or Nit at most. For example, anchor Important in effects such as broken behavior, leaked data, or blocked rollback. This mechanism belongs to the research-preview Team/Enterprise service; see 4.1.1 for its availability exclusions and the local-command distinction.
Define every level with an effect and an example
The following rubric illustrates a review whose low level includes factual comment contradictions. The inline examples are abbreviated code fragments, not complete programs; the context column supplies the assumptions needed to classify them:
| Level | Concrete effect | Code example | Context that makes it fit |
|---|---|---|---|
| Critical | A reachable defect exposes protected data or enables unauthorized destructive changes | return load_account(request.account_id) | The handler returns private account data to any signed-in caller and no surrounding check verifies ownership |
| High | A normal supported workflow produces an incorrect result or fails | total = sum(prices[:-1]) | Nonempty purchase lists are normal input, and the omitted final price makes the charged total wrong |
| Medium | A supported boundary case produces a localized incorrect result or failure without a more severe effect | first = items[0] | An empty list is a valid boundary input, and the resulting failure affects only that request |
| Low | A factual explanation misleads maintainers while runtime behavior remains correct | # returns cents above return amount_dollars | Dollars are the intended return unit, callers handle dollars correctly, and only the comment claims the wrong unit |
The context is part of the example. A function returning an account is not automatically an authorization defect: an upstream authorization check may make the call safe. Likewise, an empty-list failure is not automatically medium. If that failure stops a safety-critical workflow or corrupts state, the stated effects may put it in a higher level.
The low example also depends on review scope. If your policy excludes comment contradictions, this candidate is omitted before severity classification. Do not introduce a low bucket solely to rescue observations that the inclusion rules already reject.
Resolve overlaps using the demonstrated consequence
Tell Claude how to choose when more than one row seems applicable. In this example policy, use the highest level whose effect the evidence actually establishes. Do not rank by the most alarming hypothetical outcome. Require the finding to name the reachable path and consequence that satisfy that level.
A useful classification instruction is:
For each qualifying finding, return severity, matched severity rule,
and the demonstrated effect. Choose the highest supported level.
Do not infer a data leak, destructive change, or wider outage without
code and context supporting that consequence. If context is missing,
identify what must be checked before final classification.For a custom harness, put this rubric and its examples in the review prompt; for managed Code Review, express the corresponding Important-versus-Nit criteria in REVIEW.md. Keep the missing-context state distinct from a completed severity decision, so a reviewer can inspect an uncertain candidate without publishing an unsupported critical label. A candidate receives a severity level only after it qualifies and its context supports the effect; missing context remains a separate state.
Test consistent classifications
Use cases where the expected category and severity are known. Include representative examples for every level, plus pairs that share syntax but differ in consequence. An account loader with an ownership check and one without it should not receive the same result merely because their final call is identical.
Anthropic recommends task-specific evaluations that include edge cases. For severity, score whether a qualifying finding receives the expected level and whether its explanation cites the right effect. A label alone can match by accident while the explanation applies the wrong rule. Use code-based exact matching for fixed labels. For consequence explanations requiring complex judgment, use an LLM grader after validating it against a human-labeled sample. Anthropic advises avoiding slow, expensive human grading where possible and testing an LLM grader’s reliability before scaling it.
Key distinctions and trade-offs
Impact versus frequency: A boundary case can be rare and severe. “Edge case” is not a synonym for medium; the medium example qualifies because its consequence is localized under the stated assumptions.
Evidence versus worst case: An imaginable data leak is not a demonstrated data leak. Severity should follow a supported path, with missing evidence exposed for verification.
Rubric versus exemplar: A code example anchors a definition. It does not replace the definition or mean that every similar code fragment receives the same label.
A fine-grained rubric helps prioritize work but creates more adjacent boundaries to evaluate. Start with levels your workflow can distinguish using available evidence. Additional labels are useful only if their effects and review actions remain meaningfully different.
Architecture / decision scenario
A code-generation team uses Claude to review an account and billing change. Similar findings receive critical on one run and medium on another. The architect replaces unexplained labels with the four-level rubric and context-bearing examples.
The review identifies an account handler that lacks any ownership check, a normal billing calculation that drops the final item, an empty-list failure isolated to one supported request, and a wrong unit comment above otherwise correct code. Under the stated rubric, they receive critical, high, medium, and low respectively. Each result includes the matching effect and evidence.
A second account handler has the same final loader call but checks ownership first. It produces no authorization finding. A separate empty-list failure can interrupt the entire billing workflow, so the reviewer must classify its demonstrated wider effect rather than copy the medium example mechanically.
The team tests these contrasts on repeated reviews. The acceptance question is whether equivalent effects receive equivalent labels and changed context changes the decision appropriately, not whether Claude always uses every severity level.
Common exam traps
- Providing severity names without definitions or code examples.
- Assigning severity before checking whether the observation is reportable.
- Calling every boundary-case defect medium regardless of consequence.
- Treating a security-themed code fragment as proof of exploitability.
- Using the example’s syntax instead of its effect to classify a finding.
What to remember
Give every severity level a concrete effect, a code example, and the context that connects them. Classify qualifying defects by the highest demonstrated consequence. Test nearby cases where similar code has different effects; consistent reasoning matters as well as consistent labels.
Knowledge check
A review prompt asks Claude to label each finding critical, high, medium or low. Across similar runs, one unchecked account lookup is labelled critical and a near-identical one medium, and an empty-list crash moves between high and low. Which change most directly makes the labels consistent?
A rubric defines critical as “a reachable defect exposes protected data”. Two reachable handlers return private account data to signed-in callers and both end with return load_account(request.account_id). The first checks that the caller owns the account before that line; the second has no ownership check anywhere on its path. How should the review classify them?
A rubric’s medium level reads “a supported boundary case produces a localized failure”, illustrated by first = items[0] failing on an empty list for one request. The reviewer finds an empty-list failure that stops the entire billing run for every customer, and labels it medium. How should it be classified?
A reviewer suspects that a new endpoint could leak customer records, but the request’s routing and middleware were not included in the review input, so it cannot tell whether an authorization check runs upstream. What should the finding do?
References
- Anthropic, Prompting best practices
- Anthropic, Define success criteria and build evaluations
- Anthropic, Code Review