NVIDIANVIDIA-Certified Professional: Agentic AI2026-109.4 Deploy layered safety frameworks (filters, escalation protocols).
Compose independent layers of agent defense
Completed by the knowledge check
Learning objective
Design defense in depth: input screening, tool permissions, output validation, and sandboxing.
Core idea
Defense in depth means that a failure in one control still meets another control before it causes harm. Four classifiers inspecting the same prompt do not provide the same protection as a classifier, an independently authorized tool, a checked result, and a confined execution environment.
For an architect, the useful artifact is a control-coverage table: each harmful consequence, the boundary before it, the enforcing mechanism, and the owner who tests it. The important question is what still prevents damage when the model accepts hostile instructions. OWASP’s prompt-injection guidance combines filtering with deterministic validation, least privilege, and human approval precisely because prompt instructions alone cannot guarantee prevention.
See 7.1.1 for rail placement and 9.1.1–9.1.2 for permission and sandbox implementation. Here you compose those controls into a system whose protection survives an individual miss.
How it works
Screen every source that can introduce instructions
NeMo Guardrails activates an input self-check through rails.input.flows containing self check input, with the self_check_input prompt defining what to reject. Its output counterpart is rails.output.flows containing self check output, backed by the self_check_output prompt. NVIDIA explicitly treats the example prompts as starting points for evaluation and customization.
An input check sees the text supplied at that boundary. If a subsequent retrieval or tool call introduces an attacker-controlled document, the initial check has not inspected it. Put screening at that later boundary as well, before the content becomes reasoning context: for knowledge-base chunks, Guardrails retrieval rails under rails.retrieval.flows (see 7.1.1).
At a tool boundary, the NeMo Agent Toolkit Guardrails middleware, _type: guardrails from the nvidia-nat-security[guardrails] plugin, runs input rails on a function’s arguments in pre_invoke and output rails on its output in post_invoke. Its workflow_functions list form guards every top-level string field, but nested models and lists of models are not descended. The mapping form guards only the fields you name, such as reviews: [review] for the review string inside each element of a reviews list. A path that names a missing or non-string field raises a ValueError at function registration rather than silently guarding nothing.
Track the actual inspected field in your coverage table. A policy that checks a document title while forwarding its body leaves the principal injection surface exposed; with the list form, a body nested inside a model is exactly such a gap, and the mapping form closes it. Tests should inject into the body and verify the downstream decision, rather than merely prove that the screening function ran.
Separate tool validity from tool authority
The experimental IORails engine provides rails.tool_output.flows: [tool call validation]. It checks that the model names a declared tool and that its arguments satisfy the tool’s JSON Schema. This is useful structural enforcement, but an allowed tool with valid arguments can still request an unauthorized transfer or access another customer’s record.
Keep the permission decision in the application or downstream service, using the authenticated principal and resource scope. OWASP recommends handling privileges in code and limiting them to the task. A schema-valid account identifier supplied by the model is data to authorize, not proof of authority. The tool service owner must enforce that decision even if the prompt classifier accepts the request.
IORails tool rails support the OpenAI Chat Completions wire format through the openai and nim engines and require a Colang 1.0 configuration. They are unavailable on the default LLMRails engine. Treat engine selection as part of the deployed control; see 9.4.3 for configuration failure handling.
Validate results before their next use
IORails also offers rails.tool_input.flows: [tool result validation]. It checks a returned result’s linkage through tool_call_id, its tool name, and well-formed content. That check does not enforce a response schema, inspect content safety, or establish server-verified provenance. A structurally consistent tool message can still contain an instruction to disclose confidential data.
Apply the check that matches the next consequence. Before returning an answer, use the configured output safety policy. Before passing generated content to a program, perform deterministic format validation as OWASP recommends. Before a write, enforce the authorized operation and target. A final answer refusal cannot reverse a write already performed by a tool.
This ordering also constrains delivery. If the application promises that users receive only validated answers, hold the candidate answer until its required validation finishes. With Guardrails output rail streaming (rails.output.streaming.enabled: True), that is stream_first: False: output rails check each chunk, chunk_size tokens (200 by default), before the client receives it. The default, stream_first: True, sends each chunk first, so a blocked chunk has already reached the user. Releasing text first and recording a later rejection satisfies a monitoring objective, not that release condition. Account for the added time to first token, one rail execution per chunk, in the response budget.
Confine execution after a detection miss
For a Linux Kubernetes worker, the Restricted Pod Security Standard supplies concrete controls: securityContext.runAsNonRoot: true, allowPrivilegeEscalation: false, capabilities.drop: [ALL], and an explicit seccompProfile.type, such as RuntimeDefault. Platform owners enforce these independently of the model, for example by applying the Restricted profile to the worker’s namespace through the built-in Pod Security Admission controller. The relevant privilege-escalation, seccomp, and capabilities restrictions are Linux-specific; do not copy their meaning to Windows workloads.
Network confinement is separate. A pod is isolated for egress only when a NetworkPolicy that selects it lists Egress in policyTypes. If policyTypes is omitted, Egress is set only when the policy has egress rules, so a policy with no rules and no policyTypes does not isolate egress at all. Set policyTypes: ["Egress"] with no egress rules, or with rules for only the required destinations. A default-deny egress policy also blocks DNS, so a worker that resolves names needs an explicit allowance to the cluster’s DNS service. All of this requires a network plugin that implements NetworkPolicy. Creating the resource without an enforcing controller has no effect.
These controls limit what escaped code can do. They do not classify an answer or authorize a business transaction. Verify confinement by attempting forbidden access from the worker, including after a deliberately accepted malicious prompt.
Key distinctions and trade-offs
Compare the layers by the failure each must survive:
| Layer | Deciding condition | Mechanism | Cost | Owning Bite |
|---|---|---|---|---|
| Input screening | Untrusted text can influence planning | self check input at each relevant content boundary | Detector inference, latency, false positives | 7.1.1 |
| Tool permissions | A proposed operation can cross an authority boundary | Code-enforced principal and resource authorization, beyond tool call validation | Policy integration and denied operations | 9.1.1 |
| Output validation | A candidate result is about to reach a consumer | self check output plus deterministic format checks | Generation already spent; buffering or rejection | 7.1.1, 2.1.2 |
| Sandboxing | Generated code or a compromised tool can execute | Restricted securityContext and enforced egress NetworkPolicy | Reduced compatibility and platform operations | 9.1.2 |
Extra model checks consume tokens and can share correlated blind spots. Authorization and operating-system restrictions supply different failure modes. Prefer this independence over duplicating an identical prompt filter at several network hops.
Architecture / decision scenario
An industrial-maintenance assistant analyzes supplier manuals and generates diagnostic scripts on an on-premises Kubernetes cluster. Its design target is an eight-second response; measured generation takes four seconds and each semantic check takes 600 milliseconds. The maintenance platform team owns execution safety, while the application team owns screening and answer validation. These are workload assumptions, not product performance claims.
One proposal adds three input classifiers and lets scripts execute in the agent container. It uses 1.8 seconds for screening but leaves an accepted malicious manual able to reach the container’s permissions. Choose instead one evaluated input policy, screening of retrieved manual content, and final answer validation, alongside independently restricted tools and a separate confined Linux worker.
The coverage review exercises all four layers. Inject an instruction into a manual; force the detector to miss it; attempt a schema-valid request for an unauthorized asset; and run a script that tries forbidden egress. Finally, generate an unsafe answer and verify, with stream_first: False, that no candidate text reaches the user before output acceptance. Permission denial, sandbox denial, and answer rejection must remain effective independently. If measured latency exceeds eight seconds, reduce unnecessary detector work or narrow the supported task; removing confinement to recover latency leaves the identified execution consequence uncontrolled.
Common exam traps
- Treating several prompt classifiers as independent authorization controls.
- Assuming initial input screening covers documents retrieved afterward.
- Equating schema-valid tool arguments with permission to execute them.
- Assuming
tool result validationscreens the result’s meaning or verifies its origin. - Adding a NetworkPolicy without an enforcing network plugin, or without
Egressin itspolicyTypes. - Leaving
stream_firstat its defaultTruewhen the release condition requires validation first. - Applying final output filtering after a privileged side effect has already occurred.
What to remember
Make each layer fail for a different reason: a detector, code-enforced authorization, deterministic validation, and an operating-system boundary do not share one blind spot. Screening limits hostile influence; permissions limit authority; validation limits released results; sandboxing limits execution damage. Verify that each still protects the system when another layer fails.
Knowledge check
A billing agent can call a refund tool, and an input self-check screens user messages. The threat review assumes that the detector will sometimes miss an injected instruction. Which added control still prevents an unauthorized refund in that case?
tool call validation against the refund tool’s JSON Schemaself check output rail on the agent’s final answerA research agent passes user questions through self check input, then retrieves knowledge-base chunks that outside contributors can edit. A red-team chunk carries an instruction that the agent follows. Which change screens that content before it becomes reasoning context?
self_check_input prompt for user messagesself check output rail on the final answertool result validation on returned messagesrails.retrieval.flowsA NeMo Agent Toolkit function returns a Product whose reviews field is a list of models, each with a review string. The Guardrails middleware names the function in the list form of workflow_functions, and an injection placed inside a review reaches the agent unscreened. What closes the gap?
reviews: [review] for the functionpre_invoke argumentsself check input for the user message that triggered the callreview is skipped. The mapping form guards only the named fields, traverses the nested path and checks each list element in its own rail call. Repeating the list form changes nothing, pre_invoke inspects the arguments rather than the returned reviews, and the user message is not where the injection entered.A clinical-notes assistant streams answers with Guardrails output rail streaming enabled. The compliance owner requires that users never see text that an output rail later blocks. Which setting meets that requirement?
stream_first: True , with each rejection loggedchunk_size: 50 , with stream_first at its defaultstream_first: False , buffering chunks for checkingcontext_size: 100 , with stream_first at its defaultstream_first: False , output rails process each chunk before the client receives it, so blocked content is never shown, at the cost of one rail execution per chunk added to time to first token. The default True sends each chunk first, so logging a later rejection satisfies a monitoring goal, not this one; changing chunk size or carried-over context leaves that release order unchanged.A sandboxed script worker must have no network egress, and the cluster’s network plugin enforces NetworkPolicy. The team applies the only NetworkPolicy selecting the worker pods; it sets no policyTypes and lists no rules, yet the worker still reaches external hosts. What change isolates its egress?
ingress rule list to the policypolicyTypes: ["Egress"] on the policyseccompProfile.type: RuntimeDefault on the podpolicyTypes is omitted, Ingress is always set but Egress is set only if the policy has egress rules, so this policy never applies to egress. Listing Egress with no egress rules denies all egress from the selected pods. Ingress rules govern the opposite direction, and seccomp and the Restricted profile confine what the process can do on the node, not where it can connect.References
- NVIDIA, LLM Self-Check
- NVIDIA, Tool Calling
- OWASP, LLM01 Prompt Injection
- NVIDIA, Output Rail Streaming
- NVIDIA, Middleware
- Kubernetes, Pod Security Standards
- Kubernetes, Network Policies