NeMo Agent Toolkit Safety and Security

Why it matters

A defense earns trust only when the same attacks run against the system before and after it. This example measures that difference on an agent with tools, not on a bare model.

What to practise

Run nat red-team with red-teaming.yml and --reps 5. Its scenarios inject payloads through _type: red_teaming middleware, and an evaluator scores whether each attack succeeded. Read the attack success rate (ASR) for each scenario, such as competitor redirect, data exfiltration and denial of service. Then rerun with red-teaming-with-defenses.yml, which changes only the base_workflow, and study the output_verifier, pre_tool_verifier and content_safety_guard middleware, including fail_closed and action. Compare the overall risk scores.

How it fits

The defense layers map to 9.4.1. The matched before-and-after runs are 9.4.2’s method, and the fail_closed and action pair is the decision that 9.4.3 makes explicit.