NVIDIANVIDIA-Certified Professional: Agentic AI2026-106.1 Implement retrieval pipelines (RAG, embedded search, hybrid approaches).
Plan, rewrite, and verify agentic RAG retrieval
Completed by the knowledge check
Learning objective
Implement agentic RAG: retrieval as a tool, query decomposition and rewriting, and self-correction.
Core idea
Agentic retrieval lets observations change what the system searches next. Instead of retrieving once and hoping the context is sufficient, the workflow asks focused questions, examines partial answers, and searches again for specific missing information. Each additional search should resolve a named gap rather than repeat a broad question with different wording.
Retrieval as a tool gives the agent a callable search capability; it does not require unrestricted control over the database. Query rewriting repairs how a question is expressed, decomposition separates what must be learned, and self-correction responds to an observed failure. They solve different problems and consume different budgets.
How it works
Expose retrieval as an agent-selected tool
NeMo Agent Toolkit configures providers in retrievers: with _type: nemo_retriever or _type: milvus_retriever and top_k. The documented NIM provider limits top_k to 1–50. A function with _type: nat_retriever references the provider through its retriever field and exposes it as an LLM tool. Set topic to specialize its generated tool description, or description to override that description; these tell the agent when searching this corpus is useful.
raise_errors defaults to true. With false, retrieval exceptions are logged and the tool returns an empty result, which an agent can mistake for “no documents.” Keep errors visible when the agent must distinguish failed search from an empty corpus result. Use the registered _type: nat_retriever; the documentation’s YAML example says retriever_tool, but its config class and implementation register nat_retriever.
Choose a planned retrieval graph
NVIDIA RAG Blueprint implements Agentic RAG as a LangGraph plan-and-execute pipeline beside its standard retrieve-then-generate chain. Its initial retrieval uses the standard /search path, including vector retrieval and reranking, before planning. Task execution then runs a retrieve-answer-retry cycle. Unlike an agent selecting the Toolkit tool, this graph defines the planning and retrieval stages.
The graph chooses an empty plan when initial context suffices, an answer plan, or scope discovery before answer planning. Independent tasks run concurrently; synthesis combines their sub-answers with initial context.
Enable this path per request with agentic=true in POST /v1/generate, together with use_knowledge_base=true. ENABLE_AGENTIC_RAG supplies the deployment default when the request omits agentic or sets it to null. Prefer an explicit request choice when direct lookups and multi-hop research have different latency allowances.
Rewrite references without changing the question
A follow-up such as “Does that exclusion apply to its renewal?” depends on conversation context. The Blueprint’s ENABLE_QUERYREWRITER makes an additional LLM call to produce a standalone query. CONVERSATION_HISTORY must be greater than zero; with zero history, rewriting is skipped with a warning. Enabling a flag without its required input does not repair an unresolved reference.
Inspect the rewritten query for invented policy versions or incorrectly resolved entities before diagnosing a wrong-subject answer.
The lower-cost alternative, MULTITURN_RETRIEVER_SIMPLE, concatenates history with the question. It applies when rewriting is disabled; rewriting takes precedence when both are enabled. Concatenation avoids the extra rewriting call but can carry irrelevant turns into retrieval. Choose based on the actual conversational reference problem, not just whether history exists.
Decompose independent facts and dependent hops
Decomposition turns one question into focused subquestions. “Which renewal terms changed, and how do they affect this claim?” requires finding the relevant versions, identifying changes, and applying them to the stated claim facts. Rewording the entire sentence once does not replace those distinct information needs.
The standard Blueprint chain offers ENABLE_QUERY_DECOMPOSITION=true with MAX_RECURSION_DEPTH to bound its iterative processing; the documentation’s configuration example uses 3. Standard decomposition supports only one collection; multi-collection queries are unsupported when it is enabled. Agentic RAG supports multi-collection retrieval and does not use this standard-chain feature.
Agentic RAG tasks run at one parallel level, without a DAG or depends-on construct. Do not place a subtask that needs another task’s answer beside it and assume parallel execution preserves the dependency. Its scope-discovery phase can establish what to search before answer planning. For arbitrary dependent workflows, choose an explicit orchestration design; see 5.2.2 and 5.3.1.
Correct a diagnosed gap and stop
Within an Agentic RAG task, a partial answer can cause the seed-query generator to reformulate the next search for missing information. After synthesis, AGENTIC_VERIFICATION_ENABLED=true enables a check for coverage gaps, vague claims, and wrong-subject drift. A failed verification creates targeted follow-up retrieval tasks and another synthesis. Verification runs once; the documented path has no nested verification loop.
Distinguish that mechanism from the standard chain’s ENABLE_REFLECTION. Standard self-reflection checks context relevance and response groundedness on a 0–2 scale, uses CONTEXT_RELEVANCE_THRESHOLD and RESPONSE_GROUNDEDNESS_THRESHOLD, and bounds refinement with MAX_REFLECTION_LOOP. Its default maximum is three. Low context relevance causes query rewriting and retrieval; low groundedness causes response regeneration emphasizing the context.
The Agentic RAG path does not use standard Self-Reflection, NeMo Guardrails, or VLM Inference. Query rewriting and reranking are supported. Standard reflection’s groundedness checks do not support response streaming. Choose the execution path with those limits in view instead of assuming every enabled feature composes.
Choose per-role models with AGENTIC_PLANNER_LLM_MODEL, AGENTIC_TASK_LLM_MODEL, AGENTIC_SEED_GEN_LLM_MODEL, and AGENTIC_SYNTHESIS_LLM_MODEL. Trial smaller task or seed-generation models against retrieval quality; an empty role model falls back to the planner, then the main RAG LLM. Per-request model and llm_endpoint override every role, so omit them to preserve role-specific choices.
AGENTIC_CONTEXT_MAX_TOKENS defaults to 100000; chunks beyond that context budget are truncated. The service owner watches the separate agentic_ Prometheus metrics for retrieval calls, task outcomes, stage latency, LLM usage, and verification behavior under load. Compare these with the empty-plan path before paying for planning on direct lookups.
Key distinctions and trade-offs
Choose the operation that addresses the missing information:
| Operation | Deciding condition | Mechanism | Cost or boundary |
|---|---|---|---|
| Retrieval tool | The agent chooses whether to search a corpus | retrievers, functions, _type: nat_retriever, description | Tool errors versus empty results via raise_errors |
| Rewriting | The question depends on conversational references | ENABLE_QUERYREWRITER, CONVERSATION_HISTORY | Extra LLM call; history is required |
| Decomposition | Several facts must be collected and combined | Agentic planner, or standard ENABLE_QUERY_DECOMPOSITION | Standard decomposition is single-collection; agentic supports multiple collections |
| Self-correction | A partial answer or verification exposes a specific gap | Seed-query generation, AGENTIC_VERIFICATION_ENABLED | Added latency; verification runs once |
Architecture / decision scenario
An insurance research assistant has a two-second target for direct clause lookup and a separate 12-second allowance for cross-policy comparisons. In an illustrative trial, standard RAG completes direct lookups in 1.4 seconds, while agentic processing takes 4.8 seconds. For comparisons across two policy collections, standard RAG covers all required clauses in 67% of cases; agentic processing reaches 91% in 9.2 seconds.
The application owner rejects enabling Agentic RAG globally. Direct requests set agentic=false; comparison requests set agentic=true and enable verification in the configured deployment. Conversational comparisons also retain history and enable rewriting to resolve policy references. The retrieval owner checks whether subtasks are independent before relying on parallel execution; scope discovery establishes the policy set first.
The tempting proposal is to enable decomposition and reflection together with every agentic request. The engineer rejects it: standard decomposition cannot search both collections, and the agentic path does not use those standard-chain features. A vague or wrong-subject claim that fails verification triggers targeted follow-up retrieval and a second synthesis, once. The claims specialist reviews support for generated conditions because this path does not run standard groundedness reflection, and owns the final applicability decision.
Common exam traps
- Treating query rewriting as multi-fact decomposition.
- Enabling rewriting while passing zero conversation history.
- Assuming agentic subtasks form an arbitrary dependency graph.
- Combining standard decomposition and reflection flags as if Agentic RAG used them.
- Repeating retrieval for an answer-generation defect or regenerating forever from missing sources.
- Treating a reflection score or completed verification as proof of factual correctness.
What to remember
Expose retrieval as a callable operation, preserve meaning when rewriting, separate the facts needed for decomposition, and correct a specific observed gap. Select one supported Blueprint execution path and bound its cost with per-role models, AGENTIC_CONTEXT_MAX_TOKENS, and the single verification pass. More searches help only when they find missing information worth their cost.
Knowledge check
A Blueprint deployment sets ENABLE_QUERYREWRITER=true, but the follow-up “Does that exclusion apply to its renewal?” still retrieves passages about the wrong policy, and the logs show a warning that rewriting was skipped. CONVERSATION_HISTORY is 0. What should the team change?
CONVERSATION_HISTORY above zeroMULTITURN_RETRIEVER_SIMPLE alongside the rewriter flagENABLE_QUERY_DECOMPOSITION to split the follow-up questionvdb_top_k so the referenced policy enters the candidatesCONVERSATION_HISTORY=0 the Blueprint skips rewriting with a warning. Simple concatenation applies only when rewriting is disabled, decomposition separates distinct information needs rather than resolving references, and more candidates for an unresolved query are still candidates for the wrong subject.An insurance assistant has a two-second target for direct clause lookups and a 12-second allowance for comparisons across two policy collections. The deployment’s ENABLE_AGENTIC_RAG default is false. In testing, the agentic path improves comparison coverage within the allowance but takes 4.8 seconds on direct lookups. How should the team route requests?
ENABLE_QUERY_DECOMPOSITION on every requestagentic=true only on comparison requestsENABLE_AGENTIC_RAG=true as the deployment defaultagentic=true with self-reflection on every requestagentic field lets comparisons take the planned path while direct lookups stay on the faster standard chain. A global default puts every lookup on the slower path, standard decomposition supports only one collection, and the agentic path does not use standard self-reflection.A team sets AGENTIC_TASK_LLM_MODEL to a smaller model to cut the cost of agentic requests, but usage metrics show every role still calling the large model. Their client sends agentic=true and a model value on each POST /v1/generate request. What explains the result?
model and llm_endpoint values override every agentic role, so the client must omit them for role-specific models to apply. The deployment flag only supplies a default when the request omits agentic , and the fallback order applies only to a role left empty.A NeMo Agent Toolkit agent uses a nat_retriever tool over Milvus with raise_errors: false. During a database outage, the agent tells users that the knowledge base has no documents on their topic. Which change lets the agent distinguish a failed search from an empty result?
top_k so searches finish before timeoutsdescription so the agent knows when to search_type from Milvus to nemo_retrieverraise_errors: true so failed searches surface as errorsraise_errors set to false, the tool logs a retrieval exception and returns an empty list, which the agent cannot tell apart from a search that found nothing. The default true raises the error instead. A description changes when the agent calls the tool, not what a failure looks like, and another provider would hide its failures the same way.References
- NVIDIA, Agentic RAG for NVIDIA RAG Blueprint
- NVIDIA, Multi-Turn Conversation Support for NVIDIA RAG Blueprint
- NVIDIA, Query Decomposition for NVIDIA RAG Blueprint
- NVIDIA, Enable Self-Reflection for NVIDIA RAG Blueprint
- NVIDIA, Retrievers in NVIDIA NeMo Agent Toolkit
- NVIDIA, Registered retriever tool implementation