Skip to lesson

NVIDIANVIDIA-Certified Professional: Agentic AI2026-100 Prerequisites

Assign each NVIDIA component its role

Completed by the knowledge check

Learning objective

Distinguish the roles of NVIDIA NIM, NeMo Agent Toolkit, NeMo Guardrails, Triton Inference Server, and TensorRT-LLM.

Core idea

These names describe different layers of an agentic system. NIM supplies inference microservices; NeMo Agent Toolkit connects workflow components; NeMo Guardrails applies programmable checks and conversation controls; Triton serves models through backends; TensorRT LLM optimizes model execution.

Choose the component whose responsibility matches the problem. Installing a model server does not decide an agent’s next tool, and a workflow library does not by itself determine the numerical kernels that execute a model.

How it works

NVIDIA NIM packages inference for deployment. The NIM for LLM and VLM overview describes validated containers, curated weights, and integration with inference engines. Its scope is an operational model-inference service, rather than the complete business workflow around that service. See 7.2.1 for running and calling a NIM and 7.2.2 for selecting profiles and backends. The overview describes a vLLM-based offering; profile backend tags vllm, sglang, and trtllm must match the inference engine in the selected container image.

NeMo Agent Toolkit expresses application workflows and their dependencies. In its configuration, functions defines tools and other functions, llms defines model-client instances, and workflow selects the entry function. A component’s _type selects its registered implementation. These are application composition decisions: which tool to expose, which model connection to use, and which workflow to execute. See 7.3.1 for configuration and 7.3.2 for framework integration.

NeMo Guardrails provides programmable controls at interaction stages. Input rails check incoming content; retrieval rails check retrieved context; dialog rails steer conversations; execution rails check tool interactions; output rails check responses. This places checks around behavior rather than selecting an inference kernel. See 7.1.1 for the rail model and 7.1.2 for configuration and integration. For the separate role of application authorization, see 0.10.

Triton Inference Server hosts models and routes inference through model backends. It exposes inference protocols, supports concurrent model execution and batching, and manages model deployment. This serving layer is distinct from agent workflow orchestration even when both contain multiple processing stages. See 7.4.1 for the Triton and TensorRT LLM integration.

TensorRT LLM accelerates and optimizes inference on NVIDIA GPUs. Its Python LLM API supports model execution and integrates with serving systems including Triton. Quantization, attention-cache management, and in-flight batching belong to its execution responsibilities; see 0.9 for the prerequisites and 7.4.2–7.4.3 for their configuration and evaluation. Triton and TensorRT LLM can work together: the serving system and execution library are different responsibilities, not mutually exclusive products.

Key distinctions and trade-offs

A packaged NIM reduces the integration work needed to operate a supported inference service. Choosing Triton and an execution backend directly gives the team responsibility for assembling and operating that serving arrangement. Neither choice determines the application’s tools or approval rules.

Likewise, Toolkit composition and Guardrails checks complement each other. One connects the workflow; the other constrains selected interactions. Choose checks for the stages your application actually uses rather than assuming a product name implies complete protection.

Architecture / decision scenario

A retailer has a three-person team delivering a customer assistant in six weeks. It needs catalog retrieval, order-status tools, and content checks, with a measured p95 response target of three seconds. A proposed plan starts by assembling a custom Triton and TensorRT LLM stack, although an approved NIM already passes that target on the team’s workload.

The technical lead chooses the approved NIM for inference, Toolkit configuration for workflow composition, and Guardrails checks at the relevant interaction stages. The platform owner operates the inference service; the application owner owns tools and workflow behavior. The team keeps the custom serving proposal for a demonstrated execution or serving requirement that justifies its integration cost. These staffing, schedule, and latency figures are scenario constraints, not product performance promises.

Common exam traps

  • Treating NeMo Agent Toolkit as an inference engine.
  • Treating NeMo Guardrails as a replacement for tool permissions.
  • Treating Triton and TensorRT LLM as interchangeable names.
  • Assuming every agent needs all five components deployed separately.

What to remember

Match responsibility to layer: packaged inference, workflow composition, programmable checks, model serving, and optimized execution. Validate the specific combination instead of selecting products by name alone.

Knowledge check

References