AnthropicClaude Certified Architect – Professional2026-105.5 Address ethical AI considerations (bias, fairness, transparency)

Trace bias before comparing group outcomes

Completed by the knowledge check

Learning objective

Identify sources of bias — model outputs, retrieved data, prompts, evaluation sets, and proxy variables — and measure outcomes across relevant user groups.

Core idea

Segmented accuracy exposes uneven performance. Fairness analysis adds a different responsibility: decide which differences harm people, whether the decision criteria are justified, and whose outcomes the evaluation overlooks.

NIST AI RMF 1.0 distinguishes systemic, computational and statistical, and human-cognitive bias. None requires discriminatory intent. A Claude system can reproduce an unfair institutional rule accurately, retrieve a prejudiced precedent faithfully, or receive favorable evaluation scores because its labels encode that same precedent. The architect therefore owns a system-level bias assessment rather than a declaration that one model is fair.

How it works

Trace the source of the disparity

Use a bias-source register to extend NIST Generative AI Profile action MS-2.11-004, written for training and evaluation data, to the system’s input paths and outputs. The action calls for examining data coverage, intersecting groups, latent bias, and demographic proxies. The following are illustrative failure hypotheses, not claims about a particular Claude model:

Source	Failure hypothesis	Comparison that helps locate it
Model outputs	Equally qualified people receive different assessments when demographic cues change	Matched input variants under the same task configuration
Retrieved data	Historical guidance favors one population or omits another’s circumstances	Same cases with independently approved reference material
Prompts and examples	A vague criterion rewards social similarity rather than task eligibility	Original rubric versus a criterion-specific revision
Evaluation sets	Past approvals become unquestioned labels; excluded groups are missing	Expert-adjudicated labels and a coverage inventory
Proxy variables	School, postcode, dialect, or an employment gap substitutes for a demographic attribute	Criterion review and controlled proxy substitutions

The register records the suspected path, affected population, potential harm, diagnostic comparison, and owner. Separate a model-generated association from a rule the organization explicitly supplied. Changing model cannot repair a policy that still instructs every model to penalize an irrelevant career break.

A proxy is contextual. A location may be relevant to service availability yet inappropriate as a substitute for an individual’s reliability. Require the domain owner to justify the actual decision criterion. Removing an explicit demographic field does not establish that its correlated signals disappeared. Keep demographic audit data under a separately approved access and retention arrangement; compliance and legal owners decide whether and how it can be collected.

Measure the outcome people experience

NIST action MS-2.11-002 calls for subgroup assessments covering both quality of service and allocation of services or resources. A fluent answer can still allocate access unfairly. Choose measures that follow the harm: incorrect exclusion, unnecessary escalation, unsupported statements, waiting time, or inability to complete the workflow.

Define the denominator for each measure. Selection rate is selected people divided by all assessed people in that group. False-negative rate is incorrectly excluded eligible people divided by independently established eligible people in that group. The second requires a defensible eligibility label; agreement with historical staff choices is insufficient when those choices are under investigation.

NIST names demographic parity, equalized odds, and equal opportunity as possible metrics where appropriate. Do not treat them as interchangeable targets. Equal selection rates ask about allocation across groups; comparisons conditional on eligibility ask whether similarly eligible people receive comparable treatment. The assessment plan states the chosen metric, its rationale, limitations, and decision owner. The organization’s context determines the fairness requirement; the framework supplies no universal acceptable gap.

Include intersections when they match plausible harm, such as language and age together. Report counts, missing group labels, uncertain eligibility, and review delays alongside rates. An apparent improvement obtained by sending one group into an indefinite review queue is a different service outcome, not demonstrated fairness.

Test sensitivity without mistaking noise for bias

Anthropic’s discrimination research describes systematically varying demographic information across decision scenarios. Borrow the comparison method rather than its measured results for Claude 2.0: those results do not establish performance for your deployed configuration.

Build matched cases that preserve relevant qualifications while changing only the attribute under test. Repeat unchanged cases as controls and repeat the matched variants to distinguish directional sensitivity from run-to-run variation. A single changed answer does not establish a reliable effect. Conversely, one name-swap test cannot clear a pipeline whose criteria systematically disadvantage carers.

Keep the matched-case report separate from observational production outcomes. Controlled substitutions test sensitivity to a chosen cue. Group outcome analysis shows what people actually experience, including retrieval, reviewer behavior, and access barriers. Domain experts must also review whether the criteria themselves belong in the task. These three views answer complementary questions.

Key distinctions and trade-offs

Accuracy versus fairness: Correctly following an unjustified criterion can increase accuracy against flawed labels while worsening harm. Anthropic’s evaluation guidance requires task-specific, measurable success criteria; fairness needs its own criteria and independently reviewed labels.

Outcome disparity versus demonstrated cause: A group gap starts an investigation. Differences in eligibility, missing data, workflow access, and model treatment require separate examination before attributing the cause or choosing a repair.

Coverage versus privacy: More detailed subgroup analysis can expose hidden failures but also create sensitive records and small identifiable groups. Document insufficient coverage rather than inferring sensitive attributes from names or quietly merging away a potentially harmed group.

Architecture / decision scenario

A financial-services firm uses Claude to recommend which hardship applications need specialist assistance. Its review panel labels 1,000 applications using approved eligibility rules. Of 400 eligible applicants in group A, 20 are incorrectly excluded; of 100 eligible applicants in group B, 20 are incorrectly excluded. False-negative rates are 5% and 20%, despite a favorable aggregate score.

The product manager proposes switching models because overall agreement with historical staff decisions is high. The architect rejects model selection as the first diagnosis: the historical labels and actual decision criteria remain untested. The service owner requires an investigation whenever the eligible-group exclusion gap exceeds the scenario’s five-percentage-point review threshold; that threshold is organizational policy, not a regulatory safe harbor.

The team inspects generated recommendations, retrieves the exact guidance supplied for each case, audits prompt examples, and independently rechecks evaluation labels. It finds an instruction rewarding uninterrupted employment and tests that criterion and associated career-gap proxies with matched cases and unchanged controls. Production group breakdowns remain necessary even if the matched tests improve.

The service owner keeps affected recommendations under specialist review while the evaluation lead documents label uncertainty and subgroup coverage. Compliance approves the audit-data arrangement. This choice costs reviewer capacity, but changing the model alone would leave the identified prompt and data paths intact. Mitigation selection follows in 5.5.2.

Common exam traps

Treating strong aggregate accuracy as a fairness assessment.
Assuming removal of protected fields removes demographic proxies.
Using historical decisions as unquestionable eligibility labels.
Declaring bias from one changed response without repeated controls.
Treating equal selection rates as the only defensible fairness objective.
Measuring generated text while ignoring who receives access or waits longer.

What to remember

Trace model, retrieval, prompt, evaluation, and proxy effects separately. Measure group and intersectional outcomes using harm-specific denominators, justified criteria, and visible uncertainty. Controlled variants identify sensitivities; production outcomes and criterion review establish what those sensitivities mean for people.