NVIDIANVIDIA-Certified Professional: Agentic AI2026-109.3 Mitigate bias and toxicity in outputs.
Locate bias before choosing a repair
Completed by the knowledge check
Learning objective
Identify bias sources in data, prompts, and models.
Core idea
For an agent architect, locating bias means identifying which stage introduces a harmful difference in treatment, representation, or service quality. A courteous answer can still recommend different opportunities to equally qualified people. A toxic-language detector addresses a different question: whether text violates its content-safety policy.
NIST AI RMF 1.0 distinguishes systemic, computational and statistical, and human-cognitive bias. These categories cross component boundaries. An organization’s historical exclusion can enter a dataset, a prompt author can encode it in examples, and a model can reproduce it without an explicit instruction. You need a causal investigation before choosing a prompt change, corpus repair, or model adaptation.
How it works
Define the harm and preserve the failing case
Start with the intended task and the people affected. Record the observed outcome, the relevant comparison, and the legitimate reasons outcomes may differ. NIST’s Generative AI Profile describes harmful bias as including representational harms and performance disparities across subgroups or languages. It also warns that foundation-model behavior can propagate into downstream systems.
Create a bias investigation record with the request, authorized source excerpts, assembled instructions, model revision, generated answer, and the human-reviewed harm label. Associate the record with the documented fairness assessment under NIST MEASURE 2.11. Preserve only the data your investigation is authorized to use; see 9.1.3 for operational trace design.
A missing opportunity, a stereotyped description, and an unjustified refusal require different labels. If reviewers cannot agree what makes an answer harmful, resolve the task-specific rubric with the domain owner before treating a detector score as a diagnosis.
Inspect data that reaches the answer
Distinguish training data from runtime context. Training data influences model weights; a retrieval result, tool response, or retained conversation supplies facts and examples during inference. A retrieval assistant may generate an unequal answer because one group’s documents are missing even when the underlying model can answer a balanced test correctly.
For runtime data, compare source coverage, freshness, and selection by the relevant population. Inspect the actual retrieved passages, not only the size of the complete collection. A balanced repository can still produce skewed context if queries or rankings repeatedly select one subset. Corpus admission checks belong to 6.4.2; here the question is whether their accepted material supports the people and tasks in the bias assessment.
For training data you control, inspect who appears, which outcomes become preferred examples, and whether annotation instructions reward stereotypes or unnecessary refusals. For a supplied model, treat unknown training composition as uncertainty. A disparity in outputs does not establish which training records caused it. You can still measure and compare deployed behavior without claiming access to the training corpus.
Read the supplied model card’s Bias subcard for its declared metric, worst-performing feature, and mitigation status. For example, NVIDIA Nemotron Nano 9B v2 names “BBQ Accuracy Scores in Ambiguous Contexts,” identifies Age as its worst-performing feature, and lists “None” under measures taken against unwanted bias. Treat those declarations as starting hypotheses for your task-specific tests, not as measurements of your deployment.
Inspect prompts and examples
Read the full assembled prompt, including instructions, demonstrations, retrieved text, and conversation history. An instruction such as “recommend leadership tracks to candidates resembling past executives” embeds a selection rule even without hostile language. Demonstrations can encode the same rule through who receives ambitious recommendations and who receives remedial ones.
In NeMo Agent Toolkit’s ReAct agent, system_prompt overrides the system prompt, while additional_instructions adds instructions to the base prompt. The agent uses _type: react_agent and requires nvidia-nat[langchain]. Inspect the deployed values of these fields when that agent is your implementation; a change to a separate prompt file proves little if the workflow does not use it.
Use task-relevant criteria and balanced demonstrations as repair candidates. Avoid deleting all demographic context mechanically: an accessibility request can legitimately require that context. The decision is whether an attribute should affect this task’s outcome, not whether the string appears anywhere in the prompt.
Isolate model behavior with controlled comparisons
Hold the source bundle and instructions fixed while comparing model revisions. Then hold the model fixed while changing one suspected prompt or source defect. In matched requests, vary an attribute that the domain owner says is irrelevant while preserving qualifications, intent, and difficulty. Review substantive help, assumptions, and refusal behavior rather than demanding identical wording.
These comparisons support a cause hypothesis; they do not prove an entire model is unbiased. Repeated generation can produce variation, so keep generation settings recorded and test more than one example. A prompt repair that works on one name but fails on other names or dialects remains a narrow repair. See 9.3.2 for representative measurement.
Key distinctions and trade-offs
Use the source of the defect to choose the next investigation:
| Suspected source | Deciding observation | Mechanism or artifact | Cost and owner |
|---|---|---|---|
| Data | Unequal coverage or outcomes follow the selected context | Source-coverage review and NIST MEASURE 2.11 assessment | Corpus review and reannotation; data owner |
| Prompt | Disparity changes when instructions or demonstrations change | ReAct system_prompt or additional_instructions | Prompt regression testing; workflow owner |
| Model | Disparity persists on controlled context and prompts but differs across model candidates | Model card Bias subcard and fixed-input comparison | Evaluation and possible adaptation; model owner |
The categories interact. A model can amplify skewed context, and a prompt can hide a data gap by producing confident prose. Repair the demonstrated cause first, then retest the complete agent. NIST explicitly notes that mitigating harmful bias does not by itself establish fairness in every respect.
Architecture / decision scenario
An engineering firm’s employee-training assistant must propose advanced courses within a two-second response target. In a hypothetical review of 200 matched employee profiles, it proposes an advanced track for 76% of one group and 44% of another despite identical prerequisites. Its toxicity filter flags none of the answers.
The tempting proposal is a stricter output classifier. The training-program owner rejects that as the first repair because the failure is unequal opportunity guidance, and every answer is polite. The investigation separately tests data, prompts, and models: both groups receive the same course catalog; the deployed additional_instructions asks the agent to imitate historical promotion examples; removing that instruction narrows the gap to 74% versus 70%. A model comparison on the repaired prompt still finds differences in how it handles several accessibility requests.
The workflow owner replaces the historical imitation rule with explicit course prerequisites and representative demonstrations. The data owner reviews the historical labels before they enter future training, and the model owner investigates the remaining controlled failures. The team keeps the residual cases in the assessment instead of declaring the first improvement a complete repair. Buying more classifier capacity would consume latency budget without correcting the demonstrated selection rule.
Common exam traps
- Equating a low toxicity score with equal treatment or equal service quality.
- Assuming every disparity comes from pretraining rather than assembled prompts or selected context.
- Treating equal dataset counts as proof that runtime retrieval is representative.
- Removing task-relevant accessibility information to make prompts look neutral.
- Claiming a controlled comparison identifies unavailable training records.
What to remember
Define the harm, inspect the actual data and instructions, and isolate changes before assigning a repair. Use NIST MEASURE 2.11 to document the assessment and the deployed prompt fields to locate prompt behavior. Data, prompts, and model weights are distinct repair surfaces, but their effects must be retested together.
Knowledge check
A benefits-guidance assistant, built as a NeMo Agent Toolkit ReAct agent with retrieval, gives applicants from one region shorter and less specific guidance than applicants from another region with matched circumstances. The domain owner confirms that region should not change this guidance, and every answer passes the content-safety output check. Before funding a repair, the team must find which stage introduces the disparity. What should it do next?
An internal HR assistant answers from a policy repository that holds equal numbers of documents for full-time and part-time staff. Part-time employees receive less complete answers. When the team gives matched requests from both groups the same passages and prompt, two model candidates answer both groups equally well. What should the team inspect next?
A reviewer proposes stripping every disability reference from a travel agent’s assembled prompts so that they look neutral. Some travelers state that they use a wheelchair so that the agent offers accessible rooms. Which question decides whether such a reference stays in the prompt?
A team deploys a supplied model whose training data it cannot inspect. With the source bundle and instructions held fixed across matched requests, a disparity persists on the deployed model but does not appear on a second model candidate. What does this comparison support?