AnthropicClaude Certified Architect – Professional2026-105.5 Address ethical AI considerations (bias, fairness, transparency)
Layer bias mitigations from criteria to review
Completed by the knowledge check
Learning objective
Mitigate bias with prompt design, data curation, decision constraints, and human oversight.
Core idea
Choose a mitigation for the path that produces the harm. An instruction against discrimination can change model behavior, but cannot justify an irrelevant criterion, replace missing source material, or grant a reviewer authority they do not have.
Anthropic’s discrimination research demonstrates that prompt engineering can reduce measured discrimination in its tested scenarios. That is support for testing a prompt intervention, not a guarantee for another model, corpus, or workflow. The researchers also state that Anthropic does not endorse or permit automated model decisions in the high-risk use cases the study examines. Your bias-mitigation plan combines four layers and measures the complete resulting service using the analysis in 5.5.1.
How it works
Make the prompt express approved criteria
Put the approved assessment rubric in the Claude API’s system instructions and distinguish it from the applicant or case material in messages. The rubric names relevant criteria, disallowed inferences, and the treatment of missing information. For example, assess whether an applicant can meet a published shift requirement rather than infer commitment from continuous employment or writing style.
Ask for a criterion result supported by the supplied material, and permit an unresolved result when the necessary information is absent. Distinguish “not stated” from “does not qualify.” This is a task-specific prompt design choice, not a documented Claude fairness mode. The domain owner signs off on what each criterion measures before the evaluation lead measures its application.
Review examples as policy-bearing content. A set of successful examples drawn entirely from one social background can teach a preference that the general instruction rejects. Apply the Diverse rule in Anthropic’s prompting guidance: cover edge cases and vary circumstances to avoid unintended patterns. Wrap each example in <example> tags and the set in <examples> tags; the guidance recommends 3–5 examples for best results. Include cases with missing information and legitimate alternative qualifications. Evaluate revisions against the same held-out cases so a favorable result cannot come from changing the rubric and answer key together.
Curate what the decision can use
Apply NIST Generative AI Profile action MS-2.11-004 to the input inventory: record coverage, demographic proxies, and latent bias in source and evaluation data. The concrete artifact is a curation register naming each source, its approved use, excluded material, and responsible owner.
Replace obsolete or discriminatory retrieved guidance with approved criteria and preserve the revision used for each assessment. Review historical labels before reusing them as examples. A curator who removes obviously offensive wording but retains an exclusionary rule has changed presentation without repairing the decision.
Where an attribute is irrelevant, remove it before constructing the request rather than relying only on an instruction to ignore it. Test the transformation: removing names while leaving identifying headers or metadata defeats its purpose. Also test that masking does not erase relevant evidence or reduce performance for people whose documents use unfamiliar formats.
Keep authorized audit attributes separate from decision inputs. Masking the decision input and retaining appropriately controlled group data for evaluation serve different purposes. Compliance owners determine the permissible collection and retention; masking alone does not settle those questions. Do not reconstruct group membership by asking Claude to guess it.
Constrain the effect outside the model
Use a decision-constraint specification to state which actions a recommendation can trigger. NIST AI RMF MANAGE 4.1 supports documented appeal, override, incident response, and change-management mechanisms. In an application, translate the approved workflow into an enforced transition: a generated assessment remains pending until an authorized reviewer records a disposition.
A prompt that says “never reject” does not implement that transition. The service handling the final decision must check the review state and the reviewer’s authority before applying it. These are application controls, not built-in fairness settings in the Messages API.
For a Claude Code workflow that can invoke a decision tool, a synchronous command PreToolUse hook can deny that invocation by exiting with code 0 and returning hookSpecificOutput.permissionDecision: "deny", with hookEventName: "PreToolUse" and a permissionDecisionReason. This mechanism belongs to Claude Code; it is not a hook supplied by an ordinary Messages API request. The downstream service must still enforce authorization against other callers. See 5.1.3 for approval enforcement; the fairness-specific question is which outcomes require review and what information the reviewer receives.
Make oversight capable of catching exclusion
Anthropic’s Usage Policy effective September 15, 2025 requires a qualified professional to review content or decisions before dissemination or finalization for its specified High-Risk Use Cases when advice, recommendations, or subjective decisions directly affect individuals or consumers. Employment screening is among the listed cases.
Design review around the potential harm. A reviewer who sees only model-selected candidates can correct false inclusions while never seeing qualified candidates omitted by the model. Give reviewers access to the original material, applicable criteria, unresolved items, and recommendations in both directions. Record the final disposition and override reason so the assessment in 5.5.1 can include the human stage.
Recommendations in both directions put the candidates omitted by the model in front of the reviewer.
Unequal review delays remain a fairness outcome to measure, even when holding an application prevents an unsupported decision. See 5.3.4 for sizing review capacity and 5.3.5 for measuring whether reviewers catch errors.
Key distinctions and trade-offs
The four layers solve different problems:
| Layer | Main benefit | Residual risk or cost |
|---|---|---|
| Prompt design | Makes approved criteria and uncertainty explicit | Instructions can fail and criteria can still be wrong |
| Data curation | Removes unjustified source rules and improves coverage | Curation costs recur as material changes |
| Decision constraints | Prevents an unreviewed recommendation becoming final | Enforcement and exception paths need implementation |
| Human oversight | Adds domain judgment and corrective authority | Capacity limits, inconsistency, and automation bias remain |
Evaluate the combined workflow, then use controlled comparisons to locate which intervention earned its cost. A smaller group gap achieved by routing all difficult cases into an unmanaged backlog is not a successful mitigation.
Architecture / decision scenario
A retailer receives 2,000 seasonal applications a week and has capacity for 500 detailed recruiter reviews. A pilot flags 800 applications as unresolved, with career-break applicants disproportionately affected. Operations proposes keeping the prompt instruction “be unbiased” and allowing automatic rejection of unresolved cases to meet the staffing deadline.
The architect and recruiting owner instead remove employment-continuity scoring, curate role examples and retrieved criteria, and mask irrelevant personal fields before constructing requests. They keep a criterion-specific prompt with an explicit missing-information outcome. The decision service prevents any final disposition, shortlist or rejection, without a recorded qualified review; a Claude Code integration also denies attempts to invoke the finalization tool prematurely through PreToolUse.
The recruiting owner narrows rollout to the volume the team can meaningfully review and adds reviewer capacity before expansion. Recruiters inspect excluded and selected applications, and the evaluation lead compares error rates and waiting times across groups after each intervention. The alternative meets throughput by silently making missing information disqualifying; the chosen design spends capacity on correcting the demonstrated harm and preserves accountability.
Common exam traps
- Treating an anti-bias instruction as a complete mitigation.
- Masking attributes without testing surviving proxies or lost relevant content.
- Replacing source wording while retaining the same unjustified criterion.
- Letting a model-generated status authorize its own final decision.
- Reviewing only recommended candidates and missing wrongful exclusions.
- Increasing traffic beyond meaningful review capacity.
What to remember
Repair criteria and inputs, test prompt changes, constrain final actions, and fund qualified oversight. Each layer has a distinct owner and residual failure mode. Approve the complete workflow only after measuring who benefits, who is excluded, and who waits.
Knowledge check
A hiring team’s recruiters review only the shortlist that Claude recommends before interviews are offered. An audit finds qualified applicants who never reached a recruiter. Which change lets the human stage catch this failure?
A staffing firm’s final-disposition service is called by a Claude Code workflow and also by a nightly batch job that sends Messages API requests. Policy says no rejection becomes final without a recorded qualified review. Where must that rule be enforced?
PreToolUse hook that denies the finalization tool in Claude CodePreToolUse hook belongs to Claude Code and does nothing for the batch job’s Messages API requests. A prompt rule can fail, and a review status that the model writes lets the recommendation authorize its own final decision.A lender masks applicants’ age and nationality before building Claude requests for affordability assessments. Compliance has approved keeping those attributes, under restricted access, for fairness evaluation. How should the evaluation lead measure group outcomes?
A bank’s hardship pilot receives 3,000 claims a week, and its specialists can meaningfully review 800. Claude marks 1,200 claims as unresolved because information is missing, and claimants with irregular income are overrepresented among them. Operations proposes automatically declining unresolved claims to keep pace. Which plan addresses the review shortfall while keeping final decisions under qualified review?
References
- Anthropic, Evaluating and Mitigating Discrimination in Language Model Decisions
- Anthropic, Prompting best practices
- Anthropic, Usage Policy, effective September 15, 2025
- Anthropic, Hooks reference
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile