Study guide for the NVIDIA-Certified Professional: Agentic AI exam
NVIDIA · October 2026 exam blueprint · 232 curriculum items
Start guide
0.1 Choose models by task and training
About this guide
Each bite isolates one concept or one architecture decision. Read them in order to prepare for the exam, or come back to a single bite later as a reference — the shape is the same either way, so you always know where in a lesson an answer will be.
Lessons are made to fit into the gaps between real engineering work, including the time while your coding agent is running. There is no reading-time promise attached to that, only a deliberate scope: one decision per bite.
A Lab completes when you open its official exercise — and, if it ends with a knowledge check, once you have answered that too. Anything else that carries a knowledge check — a bite, a checkpoint, a mock exam — completes when you answer every question of it; a bite without one completes when you say so. Completion only accumulates — nothing here takes it back.
Prerequisites
Outside the exam blueprint0 of 11 items completed
- 0.1 Choose models by task and trainingDistinguish pretrained capability, instruction following, and reasoning behavior when choosing a model for an agent.
- 0.2 Budget tokens across the inference phasesAccount for prompt and generated tokens, and distinguish prefill work from decode work when explaining latency.
- 0.3 Ground answers with retrieved document chunksExplain how embeddings and similarity search supply context to a generator without confusing relevance with correctness.
- 0.4 Choose who controls the next stepDistinguish an agent's observation-driven action selection from a predetermined workflow and choose where each belongs.
- 0.5 Separate instructions from observations and stateAssign each piece of an agent's information to its proper role before deciding what enters a prompt or persists.
- 0.6 Distinguish tool requests from structured answersRead JSON Schema function definitions and separate proposed tool execution from schema-constrained response generation.
- 0.7 Bound async calls and classify failuresCall an OpenAI-compatible endpoint asynchronously and distinguish transport failures, HTTP errors, and usable responses.
- 0.8 Separate images, workloads, and service addressesExplain how Docker images become running Kubernetes workloads and how Services keep callers independent of individual Pods.
- 0.9 Budget GPU memory beyond model weightsDistinguish precision, quantization, and batching when estimating whether an inference workload fits GPU memory and latency constraints.
- 0.10 Separate availability, quality, telemetry, and authorityDistinguish model serving, task evaluation, operational signals, and application permissions when assessing production readiness.
- 0.11 Assign each NVIDIA component its roleChoose NVIDIA components by separating packaged inference, workflow orchestration, programmable guardrails, model serving, and optimized execution.
Agent Architecture and Design
15% of the exam0 of 32 items completed
Design user interfaces for intuitive human-agent interaction.
- 1.1.1 Choose chat, copilot, form, voice, or inboxChoose a user interaction surface and connect it to an appropriate workflow interface without confusing streaming with background execution.
- 1.1.2 Show status, sources, and recoverable errorsTranslate workflow events into concise status, tool outcomes, source references, uncertainty, and recoverable error messages.
- LABLaunch the NVIDIA NeMo Agent Toolkit API Server and User InterfaceServe a workflow with the NeMo Agent Toolkit API server, then connect the web UI to it over HTTP or WebSocket.
Implement reasoning and action frameworks (e.g., ReAct).
- 1.2.1 Choose reactive, deliberative, or hybrid controlChoose reactive, deliberative, or hybrid control by locating which decisions require planning and which must remain deterministic.
- 1.2.2 Bound each ReAct execution pathConfigure ReAct tool access, parsing recovery, tool-error handling, and termination without confusing call counts with elapsed time.
- 1.2.3 Choose ReAct, tool calling, or ReWOOChoose ReAct, native tool calling, or ReWOO by comparing tool interfaces, observation dependencies, planning overhead, and failure recovery.
- LABAgents the Hard WayBuild one agent iteration by hand from a model client, a tool schema, a message list and manual tool routing.
- LABReWOO Agent ExampleRun and evaluate a ReWOO agent that plans every tool call up front, executes the plan, and then solves.
Configure agent-to-agent communication protocols for collaboration.
- 1.3.1 Choose MCP for tools and A2A for agentsChoose MCP for tool access and A2A for remote-agent collaboration while recognizing their overlapping deployment patterns.
- 1.3.2 Delegate work through discoverable agent cardsConfigure a discoverable A2A endpoint and distinguish task identity, progress, and output artifacts when exchanging results.
- 1.3.3 Secure and bound remote A2A callsProtect remote A2A calls with verified identity, restricted authorization, trace propagation, and explicit execution limits.
- LABMath Assistant A2A ExampleServe a workflow over A2A, discover its agent card, and call it from a per-user agent that also uses MCP and local tools.
Manage short-term and long-term memory for context retention.
- 1.4.1 Route history, episodes, and facts to storesChoose stores for active context, conversation history, episodic experience, and semantic knowledge by their scope and lifetime.
- 1.4.2 Scope memory and retire invalid factsDefine authenticated memory scopes and assign retention, correction, and invalidation policies to users and tenants.
- LABAutomatic Memory Wrapper for NeMo Agent Toolkit AgentsWrap an agent so that memory is saved and retrieved on every turn, scoped by user and conversation.
Orchestrate multi-agent workflows and coordination.
- 1.5.1 Choose who directs specialist agentsCompare supervisor, hierarchical, peer-to-peer, and agent-as-tool designs by control ownership, context isolation, and coordination cost.
- 1.5.2 Coordinate specialist execution through explicit outcomesDesign specialist handoffs, shared-state updates, cancellation, and failure propagation without mistaking partial output for successful completion.
- 1.5.3 Require measurable gains from specialist delegationCompare single-agent and multi-agent designs using quality, critical-path latency, token usage, and concurrent workload measurements.
- LABMulti-Agent Orchestration with NVIDIA NeMo Agent ToolkitBuild an orchestrator agent that delegates to data-analysis, visualization and retrieval specialists, then swap in a custom agent with human approval.
- LABBuilding Agentic AI Applications With LLMsTake NVIDIA's Deep Learning Institute course on building agentic applications with large language models.
Apply logic trees, prompt chains, and stateful orchestration for multi-step reasoning.
- 1.6.1 Validate results before advancing prompt chainsDesign prompt chains with typed intermediate results, distinguish compatibility checks from runtime validation, and stop invalid data before downstream generation.
- 1.6.2 Choose rules before delegating semantic routingChoose rule-based or LLM-based routing, compose bounded decision trees, and distinguish valid branch selection from correct and authorized execution.
- 1.6.3 Bound adaptive work inside explicit state graphsModel execution state, transitions, reducers, and termination in a workflow graph, then choose predetermined or adaptive control for each part of the task.
- LABHybrid Control Flow ExampleCompose router agents and sequential executors into one workflow and route requests through nested pipelines.
Integrate knowledge graphs to enable relational reasoning.
- 1.7.1 Preserve identity and sources in graph assertionsDesign typed graph entities and relationships that retain the identity, scope, and source needed to justify relational answers.
- 1.7.2 Validate extracted triples before graph writesSeparate parsing, type checks, entity resolution, and source verification when converting text into graph assertions.
- 1.7.3 Bound GraphRAG expansion for multi-hop answersChoose and bound a GraphRAG path that connects relevant text to explicit relationships for multi-hop answers.
- LABBuild Knowledge Graphs with txt2kgExtract triples from documents with a local LLM, store them in a graph database, and query the graph.
Ensure adaptability and scalability of the agent’s architecture.
- 1.8.1 Replace components through explicit interfacesSeparate model, tool, memory, and orchestration dependencies so configuration changes select compatible implementations.
- 1.8.2 Contain failures before they consume capacityCombine deadlines, separate concurrency pools, resource limits, and sandboxed execution to keep tool failures from spreading.
- 1.8.3 Scale replicas without copying session ownershipCombine replaceable serving replicas, shared durable state, and deferred tool and skill loading while accounting for their separate costs.
- LABModule 7 Lab — Agent Harnesses & SkillsBuild a minimal four-tool agent harness, measure its context overhead, and load portable skills lazily.
Agent Development
15% of the exam0 of 29 items completed
Engineer prompts and dynamic prompt chains for reliable performance.
- 2.1.1 Divide system prompts into role, rules, tools, examplesStructure role, constraints, tool guidance, and examples without breaking the agent's prompt interface.
- 2.1.2 Enforce answer schemas and bound their repairChoose a structured-output strategy and distinguish schema validation, semantic checks, and bounded response repair.
- 2.1.3 Assemble context from validated request stateCompose each model request from validated state, selected retrieval passages, and correctly associated tool results.
- 2.1.4 Test prompt revisions against model changesVersion prompt artifacts and compare their behavior across model changes using repeatable regression runs.
- LABBuilding RAG Agents With LLMsTake NVIDIA's Deep Learning Institute course on building retrieval-augmented agents with large language models.
Integrate generative and multimodal models (text, vision, audio).
- 2.2.1 Route visual questions to models or retrievalIntegrate image inputs with a vision-language model and choose when document questions need multimodal retrieval.
- 2.2.2 Synchronize speech within a turn budgetConnect streaming recognition, agent generation, and speech synthesis while controlling transcript finality, audio order, and time to first playback.
- LABBuilding a Voice-Powered RAG Agent with NVIDIA Nemotron ModelsBuild an agent that transcribes speech, retrieves from text and images, and checks its input and output for safety.
- LABRetriever API Usage with Multimodal Query SupportQuery an NVIDIA RAG Blueprint collection with text and an image, through its search and generate APIs.
Build and connect custom tools, APIs, and functions for external system interaction.
- 2.3.1 Make tool inputs and results unambiguousDesign tool names, descriptions, parameter schemas, and return types that distinguish valid requests from correct and authorized operations.
- 2.3.2 Constrain external actions inside validated adaptersWrap REST APIs, database queries, and local functions with explicit validation, authorization, and mutation boundaries.
- 2.3.3 Publish and select remote MCP toolsExpose NeMo Agent Toolkit functions through MCP and consume a restricted remote tool set while checking transport, errors, and protocol compatibility.
- LABAdding Tools to NVIDIA NeMo Agent ToolkitWrite, register and configure sales-data and retrieval tools for a ReAct agent in NeMo Agent Toolkit.
- LABMCP Setup and Integration with NVIDIA NeMo Agent ToolkitPublish a NeMo Agent Toolkit workflow as an MCP server and consume MCP tools in a workflow through the mcp_client function group.
Implement error handling (retry logic, graceful failure recovery).
- 2.4.1 Route failures to the right ownerClassify tool failures and choose between bounded model repair, caller exceptions, and explicit incomplete outcomes.
- 2.4.2 Fit retries inside the response deadlineCombine attempt limits, exponential backoff, jitter, and execution timeouts without multiplying retries across layers.
- 2.4.3 Isolate failing dependencies with circuit breakersConfigure circuit-breaker admission, bounded recovery probes, and target isolation without confusing breaker state with retry limits.
- 2.4.4 Recover actions without duplicating their effectsChoose approved fallback, safe replay, or compensation from a durable record of completed and uncertain side effects.
- LABMiddlewareWrap NeMo Agent Toolkit functions and LLM calls in configurable middleware for caching, timeouts and guardrails.
- LABSequential ExecutorChain three functions into a fixed pipeline with NeMo Agent Toolkit's sequential executor and choose how a failing step ends it.
Develop dynamic conversation flows with real-time streaming and feedback mechanisms.
- 2.5.1 Route streamed tokens and steps separatelyChoose SSE or WebSocket and interpret workflow output, intermediate steps, and terminal outcomes without mixing their roles.
- 2.5.2 Recover dialog turns from execution stateSeparate conversation history, execution state, and transport state when handling interruptions, reconnections, and resumed turns.
- 2.5.3 Turn session feedback into explicit branchesCapture typed user feedback, associate it with a pending interaction, and change the next conversation step within a bounded workflow.
- LABNeMo Agent Toolkit - UIConnect the NeMo Agent Toolkit chat UI to a served workflow over HTTP streaming and WebSocket, including a human-in-the-loop turn.
Evaluate and refine agent decision-making strategies.
- 2.6.1 Score tool choices and trajectories separatelySeparate tool-selection accuracy, argument correctness, and trajectory quality to identify which agent decisions need refinement.
- 2.6.2 Compare strategies on identical task casesCompare ReAct, tool calling, planning, and routing using matched tasks, execution conditions, and action-quality measurements.
- 2.6.3 Choose description, prompt, model, or RL fixesChoose tool-description, prompt, model, or reinforcement-learning changes from observed action-selection failures and verify the improvement.
- LABStandalone ATIF Evaluation with Custom EvaluatorsScore two agent trajectories with custom exact-match and tool-count evaluators, using only the NeMo Agent Toolkit evaluation package.
- LABTrain a CLI Agent with GRPO using Unsloth and NeMo GymFine-tune Nemotron-Nano-9B-v2 with GRPO against a NeMo Gym verifier so it turns requests into LangGraph CLI commands.
Evaluation and Tuning
13% of the exam0 of 21 items completed
Implement evaluation pipelines and task benchmarks to measure performance.
- 3.1.1 Turn task requirements into benchmark casesDefine measurable task outcomes and curate labeled or synthetic benchmark cases that expose consequential failures.
- 3.1.2 Calibrate judges against human task labelsConfigure task-specific LLM judging and measure whether its decisions agree with adjudicated human labels.
- 3.1.3 Separate retrieval failures from answer failuresUse faithfulness, answer relevancy, context precision, and context recall to locate distinct failures in a RAG pipeline.
- 3.1.4 Preserve inputs and outputs across evaluationsAutomate NeMo Agent Toolkit evaluation runs while retaining versioned inputs, effective configuration, and distinct execution artifacts.
- LABRAG Agent EvaluationEvaluate an IT help desk RAG agent on a fixed dataset with a rubric-based LLM judge, RAGAS metrics and custom checks, then save the run for a fair comparison.
- LABAgent Evaluation in NVIDIA NeMo Agent ToolkitRun a NeMo Agent Toolkit workflow over a dataset with `nat eval`, configure its evaluators, and read the workflow, configuration and evaluator outputs it writes.
Compare agent performance across tasks and datasets.
- 3.2.1 Isolate changes before comparing agent resultsDesign matched agent comparisons that fix datasets, scoring, and execution conditions while isolating the change under test.
- 3.2.2 Check task slices across repeated runsCompare agents by task, difficulty, and user cohort while distinguishing repeat-run variation from meaningful improvements.
- 3.2.3 Choose agents within quality and delivery budgetsCompare model and workflow configurations using task quality, complete-request latency, and cost per acceptable result.
- LABSimple LangSmith-Documentation Agent - Evaluation and ProfilingEvaluate and profile a documentation-query agent with `nat eval`, then rerun it under alternative configuration files and check what each one changes before comparing scores.
- LABReport Agent EvaluationEvaluate a web-search report agent with an LLM judge for structure, content, coverage, evidence support and writing, then compare a changed run on the same dataset.
Collect and integrate structured user feedback for iterative improvements.
- 3.3.1 Attach actionable feedback to exact responsesCapture ratings, diagnostic labels, and reviewed corrections against the response, trace, and session context they describe.
- 3.3.2 Curate feedback without hiding missing populationsBuild privacy-aware evaluation datasets from feedback while documenting selection, coverage, and limits on population-level conclusions.
Tune model parameters (e.g., accuracy, latency-efficiency trade-offs).
- 3.4.1 Tune temperature, top-p, and output ceilingsChoose temperature, nucleus sampling, and output limits from task reliability measurements rather than apparent consistency.
- 3.4.2 Trade model size, precision, and reasoning effortCompare model size, supported precision profiles, and reasoning controls against task quality, latency, and GPU constraints.
- 3.4.3 Choose deployable results from automated searchConfigure model, numeric, and prompt searches and select a measured Pareto candidate that meets production constraints.
- LABModel Selection and Parameter OptimizationUse `nat optimize` to compare models and temperatures for a chat workflow and a tool-calling agent, then add genetic prompt optimization and export the optimized config.
Analyze evaluation results to guide targeted optimization.
- 3.5.1 Attribute failed cases through their tracesAttribute failed cases to retrieval, reasoning, prompts, tools, or orchestration by joining evaluation results to execution traces.
- 3.5.2 Find bottlenecks in step token and latency profilesUse per-step token counts and execution timelines to distinguish expensive model calls from latency bottlenecks.
- 3.5.3 Prioritize repairs and verify their effectRank targeted fixes by consequence and expected benefit, then use re-evaluation to confirm repairs and expose regressions.
- LABTracing, Evaluating, and Profiling your AgentTrace a retail sales agent in Phoenix, evaluate it with RAGAS and trajectory evaluators, and profile it to find token, latency and concurrency bottlenecks.
Deployment and Scaling
13% of the exam0 of 22 items completed
Deploy and orchestrate multi-agent systems at production scale.
- 4.1.1 Split services by execution and ownershipSeparate agent execution, workflow coordination, and model inference into services with independent deployment and failure responsibilities.
- 4.1.2 Choose Compose or Helm by operating modelChoose Docker Compose or Helm for a multi-agent deployment and configure the service, dependency, and credential settings each path requires.
- 4.1.3 Recover distributed tasks from persisted progressCoordinate background agent work with bounded queueing, distinct waiting and execution timeouts, and recoverable job and graph state.
- LABNVIDIA Deep Researcher Agent BlueprintRun NVIDIA's deep research blueprint from the CLI, the web UI and Docker Compose, choosing a YAML workflow config, and evaluate it against a research benchmark.
- LABBuild and Deploy a Multi-Agent ChatbotDeploy a local multi-agent chatbot on DGX Spark with Docker Compose: a supervisor agent routing to coding, RAG and image-understanding agents through MCP tools.
Apply MLOps practices for continuous integration and continuous delivery (CI/CD) workflows, monitoring, and governance.
- 4.2.1 Require four checks before promoting agentsSeparate unit, integration, safety, and evaluation checks and require complete results before approving an agent release.
- 4.2.2 Pin the dependencies behind agent behaviorAssemble a release bundle that identifies prompts, graphs, tool interfaces, serving models, and datasets as one tested combination.
- 4.2.3 Choose canary, blue-green, or shadow rolloutsChoose canary, blue-green, or shadow delivery and connect release approval to monitored stop conditions and a complete rollback target.
Profile performance and reliability under distributed system loads.
- 4.3.1 Measure serving capacity against user deadlinesDistinguish TTFT, ITL, throughput, goodput, and concurrency when selecting an LLM serving operating point.
- 4.3.2 Find saturation with realistic workflow trafficDesign end-to-end load mixes and concurrency sweeps that reveal saturation, tail latency, and misleading benchmark results.
- 4.3.3 Trace the path that limits loaded workflowsUse distributed traces and workflow profiling to identify critical paths, fan-out bottlenecks, and failure propagation under load.
- LABProfiling with AIPerfServe a small model with vLLM and profile it with `aiperf profile`, reading request latency, time to first token, inter-token latency and throughput.
Scale deployments using containerization (Docker, Kubernetes) with load balancing.
- 4.4.1 Admit GPU Pods only when inference worksCombine GPU Operator provisioning, explicit resource reservations, and health probes to admit usable inference replicas.
- 4.4.2 Route inference by work and cache reuseChoose round-robin, load-aware, or KV-aware inference routing from workload variation, reusable prefixes, and queue pressure.
- 4.4.3 Scale workers against useful serving capacitySelect custom autoscaling signals while accounting for model initialization, GPU budgets, session affinity, and safe worker removal.
- LABInstalling the NVIDIA GPU OperatorInstall the NVIDIA GPU Operator with Helm, confirm its ClusterPolicy is ready, and run a CUDA sample Pod that requests a GPU.
- LABIntroduction to Deploying RAG Pipelines for Production at ScaleTake NVIDIA's Deep Learning Institute course on deploying retrieval-augmented generation pipelines for production at scale.
Optimize deployment costs while ensuring high availability.
- 4.5.1 Size replicas against measured latency targetsTranslate user demand into inference load and choose GPU capacity that meets latency and throughput targets.
- 4.5.2 Cut serving cost per acceptable taskCompare model size, quantization, caching, batching, and hosting choices by their cost per acceptable task.
- 4.5.3 Fund headroom for failures and recoveryChoose replica placement and recovery capacity that preserve service targets during failures without hiding their cost.
- LABSize a GPU Cluster With NVIDIA NeMo Agent ToolkitProfile a workflow across concurrency levels with `nat sizing calc`, then estimate the GPU count for a target number of users and response time.
- LABAuto Deploy with DGDRDescribe a model, its workload and its latency targets in a DynamoGraphDeploymentRequest, and let Dynamo profile the hardware and generate the deployment.
Cognition, Planning, and Memory
10% of the exam0 of 20 items completed
Implement memory mechanisms for short- and long-term context retention.
- 5.1.1 Keep conversations within a token budgetChoose conversation buffers, truncation, and rolling summaries that preserve task context within a bounded inference budget.
- 5.1.2 Reconcile facts before writing durable memoriesExtract supported facts, deduplicate repeated observations, and apply corrections without reinforcing an agent's earlier errors.
- 5.1.3 Rank useful memories and retire stale onesCombine relevance and recency in memory retrieval while enforcing decay, expiry, and targeted deletion.
- LABRedis ExamplesRun a NeMo Agent Toolkit chat workflow that stores user preferences in a Redis memory backend and recalls them by embedding search.
Apply reasoning frameworks (chain-of-thought, task decomposition).
- 5.2.1 Reserve tokens for a usable answerChoose prompting and reasoning modes while budgeting for intermediate work, final output, and downstream calls.
- 5.2.2 Verify inputs before dependent subtasksSplit complex requests into testable subtasks whose outputs satisfy the dependencies of later work.
- 5.2.3 Bound voting, critique, and reflection loopsSelect self-consistency, reflection, or self-critique while bounding their cost and checking for correlated mistakes.
- LABThe Workflow AgentRun a ReWOO plan-execute-synthesize pipeline, route a ticket through a fixed enum, and fan one input out to specialists built from a factory.
Engineer planning strategies for sequential and multi-step decision-making.
- 5.3.1 Choose upfront, interleaved, or hierarchical planningSelect upfront, interleaved, or hierarchical planning from observation uncertainty, task scope, and execution cost.
- 5.3.2 Make plan completion depend on checked resultsChoose a to-do list or dependency graph and attach explicit checks that distinguish completed actions from acceptable outcomes.
- 5.3.3 Repair the affected plan before continuingUse failures and observations to revise unfinished work while preserving valid results and bounding further planning.
- LABDeep Agents WorkshopBuild a deep agent with the deepagents library that plans with a to-do list, delegates to subagents, and runs inside explicit tool permissions and a sandbox.
Manage stateful orchestration to coordinate complex tasks and knowledge retention.
- 5.4.1 Keep execution separate from remembered knowledgeAssign execution progress, conversation context, and retained task knowledge to distinct representations with explicit authority and lifetimes.
- 5.4.2 Resume checkpoints without repeating external writesChoose durable graph checkpoints and replay-safe action placement so interrupted workflows resume without duplicating committed effects.
- 5.4.3 Isolate sessions and reconcile concurrent updatesSeparate session and user scopes, then choose local locking or conditional shared-store updates to prevent concurrent state corruption.
- LABPer-User Workflow ExampleServe a NeMo Agent Toolkit workflow whose notepad and preference functions keep separate state for each user session.
Adapt reasoning strategies based on prior experiences and feedback.
- 5.5.1 Reuse trajectories without copying their mistakesTurn reviewed agent trajectories into relevant examples and scoped instructions that improve subsequent reasoning without updating model weights.
- 5.5.2 Choose DPO or GRPO from your feedback signalChoose DPO or GRPO according to the feedback signal, trajectory structure, and cost of collecting useful comparisons.
- 5.5.3 Test adaptations for regressions, reward defects, and overfittingDetect regressions, reward defects, and overfitting when evaluating revised examples, instructions, or trained agent policies.
- LABDPO Tic-Tac-Toe: Preference Learning with NeMo CustomizerCollect scored candidate moves from tic-tac-toe games as DPO preference pairs, and submit them to NeMo Customizer for training with `nat finetune`.
Knowledge Integration and Data Handling
10% of the exam0 of 24 items completed
Implement retrieval pipelines (RAG, embedded search, hybrid approaches).
- 6.1.1 Size searchable chunks around usable contextChoose chunk size, overlap, structural boundaries, and parent expansion to retrieve precise passages without losing the context needed to answer.
- 6.1.2 Fuse lexical and semantic retrieval ranksChoose dense, BM25, or hybrid retrieval and combine candidate rankings without confusing incompatible scores or reranking stages.
- 6.1.3 Select embeddings before tightening reranker precisionSelect compatible embedding models and tune candidate retrieval, reranking, and context retention against quality and latency requirements.
- 6.1.4 Plan, rewrite, and verify agentic RAG retrievalChoose retrieval tools, query rewriting, decomposition, and bounded correction while respecting the supported NVIDIA RAG Blueprint execution path.
- LABAgentic RAG WorkshopBuild a LangGraph IT help desk agent that decides when to call a reranked knowledge-base retriever, web search over MCP, and loadable skills.
Configure and optimize vector databases for fast retrieval.
- 6.2.1 Choose a vector index by recall and capacityChoose flat, IVF, HNSW, or CAGRA indexes from retrieval accuracy, query load, memory capacity, and build costs.
- 6.2.2 Tune retrieval against filtered exact resultsTune distance metrics, search breadth, metadata filters, and GPU execution against a consistent recall and latency target.
- 6.2.3 Keep vector mutations visible and isolatedManage vector inserts, corrections, collection replacement, and tenant access without confusing acknowledgment with safe retrieval.
- LABVector Database Configuration for NVIDIA RAG BlueprintSwitch the NVIDIA RAG Blueprint between its Elasticsearch default and Milvus, and see what a custom vector database operator has to implement.
- LABMilvus Configuration for NVIDIA RAG BlueprintConfigure Milvus in the NVIDIA RAG Blueprint for GPU or CPU indexing and search, and enforce per-user access with runtime auth tokens.
Build extract, transform, and load (ETL) pipelines to integrate enterprise or client data sources.
- 6.3.1 Choose document extraction paths by modalityChoose extraction paths for text, tables, charts, and images while preserving the structure downstream retrieval needs.
- 6.3.2 Choose ingestion cadence from freshness targetsChoose batch, incremental, and event-driven ingestion using change volume, freshness targets, and recoverable document updates.
- 6.3.3 Join source identity, permissions, and ingestion outcomesPreserve source identity and authorization metadata while distinguishing accepted ingestion jobs from complete, recoverable document results.
- LABQuickstart: retriever CLIIngest PDFs with the NeMo Retriever `retriever` CLI locally, in batch or through a service, and query the result with dense or agentic retrieval.
- LABDocument Continuous Ingestion from Object StorageDeploy the NVIDIA RAG Blueprint with a Kafka consumer that indexes each document uploaded to MinIO, then query the collection it fills.
Conduct data quality checks, augmentation, and preprocessing.
- 6.4.1 Repair, strip, and deduplicate corpus textChoose text repair, boilerplate removal, and duplicate matching that reduce corpus noise while preserving consequential distinctions.
- 6.4.2 Admit documents through independent quality checksSeparate completeness, validity, freshness, and PII checks and assign a measurable admission rule and repair owner to each.
- 6.4.3 Enrich sources and validate synthetic additionsAttach usable metadata to source documents and admit synthetic examples only after structural, grounding, and coverage checks.
- LABLangGraph CLI Synthetic Dataset GenerationGenerate tool-calling training pairs for the LangGraph CLI with NeMo Data Designer, filter them, and split them for training.
- LABNotebook: Metadata usage, Metadata extraction.Ingest documents into NVIDIA RAG Blueprint collections with a typed metadata schema, then filter retrieval by hand, by generated filter, and across collections.
Enable real-time access and reasoning over structured and unstructured knowledge.
- 6.5.1 Constrain SQL and tabular queriesChoose text-to-SQL or bounded tabular tools and verify query meaning, execution permissions, and result completeness.
- 6.5.2 Join authorized facts to attributable passagesCombine database results and retrieved passages while preserving access decisions, source identity, and support for each claim.
- 6.5.3 Reconcile live knowledge before answeringChoose streaming ingestion or live search and resolve stale conflicts using source time, searchable progress, and explicit authority.
- LABStreaming Data to RAG Developer ExampleDeploy a pipeline that transcribes simulated FM radio into a context-aware RAG store, and ask questions bounded by stream and time window.
NVIDIA Platform Implementation
7% of the exam0 of 24 items completed
Integrate NVIDIA NeMo Guardrails for compliance and safety enforcement.
- 7.1.1 Place each rail before its consequenceChoose input, dialog, retrieval, execution, and output rails by the data they inspect and the consequence they must prevent.
- 7.1.2 Wire Guardrails from YAML to ColangConnect config.yml activation lists, task-specific prompts, and Colang flows into a testable NeMo Guardrails configuration.
- 7.1.3 Measure NemoGuard detectors on your trafficIntegrate NVIDIA safety NIMs and assess missed harms, false positives, detector outages, and latency overhead on representative traffic.
- LABCombined Guardrails: Healthcare Patient Support ChatbotCombine PII detection, content safety and topic control rails in one NeMo Guardrails configuration and test which rail stops each request.
- LABIntegrate NeMo Guardrails with NemoGuard NIM MicroservicesDeploy an LLM NIM with the Content Safety and Topic Control NemoGuard NIMs and route chat completions through a guardrail configuration.
Deploy NVIDIA NIM microservices for high-performance inference.
- 7.2.1 Validate NIM from health to completionDistinguish container startup, model readiness, and a valid OpenAI-compatible response when bringing a NIM endpoint into service.
- 7.2.2 Match serving profiles to available GPUsSelect a NIM profile and matching backend image while accounting for GPU memory, parallelism, context length, and runtime overrides.
- 7.2.3 Choose how Kubernetes manages NIM deploymentsChoose the NIM Helm chart or NIM Operator and configure image access, cached models, GPU resources, and deployment verification.
- LABNIM for LLMs QuickstartRun a NIM LLM container, find its served model name, and confirm it with health checks and a chat completion.
- LABHelm and KubernetesDeploy a NIM LLM to Kubernetes with the nim-llm Helm chart and verify the release through its readiness endpoint.
Optimize workflows with the NVIDIA NeMo Agent Toolkit.
- 7.3.1 Wire named components into runnable workflowsDefine toolkit component instances, resolve their dependencies, and separate workflow changes from model and tool configuration.
- 7.3.2 Bring framework agents under toolkit configurationChoose a toolkit integration path that preserves an existing agent while exposing the components and traces you need to control.
- 7.3.3 Feed profiler predictions into served workflowsTurn toolkit profiling findings into bounded workflow changes and verify the selected configuration through its serving interface.
- LABProfiling and Performance Monitoring of NVIDIA NeMo Agent Toolkit WorkflowsConfigure the Agent Toolkit profiler under eval, run it with nat eval, and compare LLMs from its traces, forecasts and bottleneck charts.
- LABBringing Your Own Agent to NVIDIA NeMo Agent ToolkitMigrate an existing LangChain agent into an Agent Toolkit workflow in stages, ending with a configuration that needs no custom code.
Leverage NVIDIA TensorRT-LLM and Triton Inference Server for latency reduction.
- 7.4.1 Separate Triton serving from model executionServe a supported model through Triton's LLM API integration while assigning request handling, scheduling, and execution controls to their correct layers.
- 7.4.2 Balance prefill work against active decodingConfigure runtime batching, paged KV-cache capacity, and chunked prefill to balance first-token latency with ongoing generation.
- 7.4.3 Test precision and drafting changes separatelyApply supported quantization and speculative decoding configurations, then distinguish numeric regressions, acceptance behavior, and workload-dependent speedups.
- LABTensorRT-LLM User GuideServe a Hugging Face model through Triton's TensorRT-LLM backend with the LLM API model template and send a generate request.
- LABSpeculative Decoding with TensorRT-LLMServe a model with EAGLE-3 speculative decoding in Triton, read acceptance metrics, and benchmark the speedup with GenAI-Perf.
Manage and optimize multimodal input pipelines on NVIDIA hardware.
- 7.5.1 Route media through GPU decode and extractionChoose decoding and extraction stages for video, images, audio, and documents while respecting CPU stages and GPU support limits.
- 7.5.2 Fit visual detail within serving budgetsBalance image resolution, sampled video frames, and VLM batching using measured quality, GPU memory, and end-to-end latency.
- LABNVIDIA VSS QuickstartDeploy the VSS Blueprint's base vision agent, upload a video, ask questions about it, and generate a report.
- LABRun Your First NVIDIA Speech NIM Microservice for Automatic Speech RecognitionDeploy a Parakeet ASR NIM and transcribe the same audio in streaming and offline mode over gRPC, WebSocket and HTTP.
Run, Monitor, and Maintain
7% of the exam0 of 16 items completed
Define monitoring dashboards and reliability metrics.
- 8.1.1 Set reliability targets around completed tasksDefine measurable availability, latency, error, cost, and task-success targets with explicit populations, windows, and owners.
- 8.1.2 Join GPU health to agent outcomesBuild layered DCGM Exporter, Prometheus, and Grafana views that distinguish GPU pressure, inference delays, and failed agent tasks.
- LABSetting up PrometheusInstall the Prometheus and Grafana stack on Kubernetes, deploy DCGM Exporter, and read GPU metrics on the NVIDIA DCGM dashboard.
- LABNVIDIA NIM Operator ObservabilityInstall Prometheus, enable metrics on a NIMService, and read the NIM Operator's resource-state metrics.
Track logs, errors, and anomalies for root cause diagnosis.
- 8.2.1 Join service logs to workflow spansCorrelate service logs, inference requests, and OpenTelemetry spans without confusing request identity with causal context.
- 8.2.2 Scope anomaly detectors by population and denominatorDetect latency, token, error, and output anomalies using comparable populations and explicit investigation thresholds.
- 8.2.3 Isolate causes before changing failing componentsUse correlated traces and logs to distinguish tool, retrieval, and inference causes and verify a targeted repair.
- LABSimple Calculator with Observability and TracingRun a NeMo Agent Toolkit ReAct workflow with tracing exporters and read its span hierarchy in Phoenix and other backends.
Continuously benchmark deployed agents against prior versions.
- 8.3.1 Keep release benchmarks representative and comparableMaintain golden datasets that reflect production changes while preserving comparable regression results for every agent release.
- 8.3.2 Separate release effects from production driftCompare live agent versions with concurrent cohorts and distinguish release regressions from changes in workload and outcome quality.
Implement automated tuning, retraining, and versioning in production.
- 8.4.1 Choose tuning or training from failuresDefine production triggers that select prompt and configuration experiments or a separately evaluated model-training candidate.
- 8.4.2 Fine-tune or distill from reviewed production dataPrepare production-derived targets and choose supervised fine-tuning or teacher-student distillation with explicit data and compute constraints.
- 8.4.3 Promote reproducible releases with complete rollbackVersion model, workflow, and dataset artifacts together and promote or restore them through explicit regression and deployment checks.
- LABFine-tuning, Inference, and Evaluation for LLM tool calling with NVIDIA NeMo MicroservicesFine-tune Llama 3.2 1B Instruct for tool calling with LoRA on the xLAM dataset, then evaluate and guard the customized model.
Ensure continuous uptime, transparency, and trust in live deployments.
- 8.5.1 Preserve service through updates and outagesCombine rolling replacement, request draining, and qualified fallback models to maintain usable service during updates and dependency failures.
- 8.5.2 Communicate incident impact, limits, and verified resultsCommunicate incident impact, operating limits, and supported outcomes with clear ownership and task-specific supporting information.
Safety, Ethics, and Compliance
5% of the exam0 of 20 items completed
Design and enforce system security and audit trails.
- 9.1.1 Constrain tool authority and isolate credentialsSeparate tool exposure, downstream permissions, and credential handling so an agent can perform only authorized operations.
- 9.1.2 Confine code, files, and outbound trafficChoose sandbox controls that restrict generated code, filesystem access, and network egress without mistaking observation for enforcement.
- 9.1.3 Make action and approval records verifiableDesign correlated action and approval records with durable collection and integrity verification that reveals unauthorized changes.
- LABAgent Safety with NemoClawPut an OpenClaw agent in a NemoClaw sandbox, test its network, filesystem, process and inference boundaries, and red-team it with a safety evaluation suite.
- LABUsing Authentication in the NeMo Agent ToolkitConfigure an OAuth 2.0 authorization-code provider so a NeMo Agent Toolkit tool calls a protected API with the user's own token.
Integrate compliance guardrails (privacy, enterprise policy).
- 9.2.1 Redact PII before each data exposurePlace PII detection and masking on prompts, generated answers, retrieved chunks, and telemetry without assuming one rail protects every copy.
- 9.2.2 Enforce topic scope and storage lifetimesTranslate enterprise policy into a topic-control rail and independently enforced retention settings with named operational owners.
- LABPII Detection for Insurance Claim AnonymizationMask PII in insurance claim narratives with the GLiNER rail in NeMo Guardrails, then measure its recall and over-masking.
- LABTopic Control for an IP-Law Research AssistantScope an assistant to IP law with the NemoGuard topic-control rail, then evaluate it on on-topic, adjacent-legal and unrelated prompts.
Mitigate bias and toxicity in outputs.
- 9.3.1 Locate bias before choosing a repairTrace harmful disparities through source data, prompt construction, and model behavior to assign a targeted repair.
- 9.3.2 Measure missed harms and overblocking separatelyBuild representative policy tests and distinguish harmful outputs that escape from legitimate requests that get blocked.
- 9.3.3 Choose prompting, classification, or safety trainingMatch bias and toxicity failures to prompt repairs, content-safety classifiers, or safety post-training while preserving useful assistance.
- LABContent Safety for an Online-Gaming CommunityModerate gaming chat with the NemoGuard content-safety rail and measure missed harassment against over-blocked banter.
Deploy layered safety frameworks (filters, escalation protocols).
- 9.4.1 Compose independent layers of agent defenseCompose input screening, tool permissions, output validation, and sandboxing so a missed detection does not become an authorized harmful action.
- 9.4.2 Compare defenses against matched adversarial attemptsDesign prompt-injection and jailbreak tests that compare defended workflows with a baseline while preserving attack coverage and legitimate task completion.
- 9.4.3 Stop uncertain actions and escalate by riskDefine safe outcomes for detector failures and review delays, then route unresolved high-risk requests to human reviewers without granting unchecked authority.
- LABNeMo Agent Toolkit Safety and SecurityRed-team a retail agent with NeMo Agent Toolkit, then rerun the attacks against defense middleware and compare the results.
- LABSecuring Agentic AI Developer Day: Evaluating Guardrailed Systems and Models with `garak`Run garak probes against a NeMo Guardrails configuration, compare the report with the unguarded model, and patch a jailbreak that gets through.
Ensure compliance with licensing and regulatory standards.
- 9.5.1 Check permissions for every shipped artifactAssess model, dataset, and software permissions against the intended use and preserve the obligations attached to each artifact.
- 9.5.2 Connect agent risks to accountable controlsClassify an agent's intended use, distinguish legal duties from voluntary risk guidance, and maintain records that connect obligations to tested controls.
Human-AI Interaction and Oversight
5% of the exam0 of 13 items completed
Build intuitive UIs with user-in-the-loop interaction.
- 10.1.1 Make review controls change execution stateConnect pause, approve, edit, reject, and resume controls to explicit workflow transitions and correlated user responses.
- 10.1.2 Keep review in session or park it as a taskChoose synchronous review or asynchronous task review from human response time, execution deadlines, and resumable workflow support.
- LABSimple Calculator - Human in the LoopRun a NeMo Agent Toolkit calculator agent that stops at its iteration limit and asks a person whether to continue.
Design structured feedback loops that guide iterative agent improvements.
- 10.2.1 Route feedback to a testable repairDefine diagnostic feedback categories that assign an improvement owner, a specific intervention, and a measurable closure condition.
- 10.2.2 Carry production feedback into re-evaluated releasesTurn adjudicated production feedback into bounded experiments, re-evaluated release candidates, and verified feedback closure.
Implement transparency mechanisms (explainable reasoning, decision traceability).
- 10.3.1 Explain actions with checkable supporting sourcesBuild concise action explanations from verified outcomes and cited sources while keeping raw model reasoning out of user responses.
- 10.3.2 Preserve observed decisions for later auditDistinguish recorded execution from generated explanations and collect decision records with explicit coverage and content limits.
- LABTracing Guardrails QuickstartTrace NeMo Guardrails requests to a local file, then compare when each rail ran under sequential and parallel input rails.
Enable human oversight and intervention for accountability and trust.
- 10.4.1 Match approval to action consequencesChoose human approval requirements from action impact and reversibility, and bind each decision to the operation that executes.
- 10.4.2 Stop agent runs and reconcile their effectsDistinguish interrupts, overrides, and cancellation, and reconcile completed operations before declaring a workflow safely stopped.
- 10.4.3 Assign reviewers with authority to interveneDefine role-based oversight, escalation ownership, and review records that connect human decisions to authorized agent actions.
- LABA Simple Jira Agent that Extracts POR and creates ticketsRun a NeMo Agent Toolkit agent that extracts work items from a requirements file and asks for approval before it creates Jira tickets.
- LABMeet the Bash AgentBuild a LangGraph bash agent whose every shell command needs your confirmation, and observe how it behaves at its permission limits.