NVIDIANVIDIA-Certified Professional: Agentic AI2026-100 Prerequisites
Ground answers with retrieved document chunks
Completed by the knowledge check
Learning objective
Explain embeddings, vector similarity, and the basic retrieve-augment-generate pattern.
Core idea
An embedding encodes an input as a numeric vector. A retrieval system compares a question’s vector with document vectors to find related material. Retrieval-augmented generation (RAG) then supplies selected material to a language model as context for an answer.
The embedding model finds candidates; the generation model writes the response. Similarity measures relevance in a representation space. It does not establish that a passage is true, authorized for this reader, or sufficient to answer the question.
How it works
NVIDIA’s NeMo Retriever Embedding NIM documentation separates offline knowledge-base encoding from online question answering. During ingestion, documents are split into chunks. An embedding model encodes each chunk, and a vector database stores its vector with the corresponding document and metadata. See 6.1.1 for chunking choices.
At query time, the system encodes the question into a compatible embedding space and searches the stored vectors. For models that operate in passage or query mode, NeMo Retriever Embedding NIM requires input_type: "passage" during indexing and input_type: "query" for questions. The wrong mode reduces retrieval accuracy. The retrieved chunks become context alongside the question in a generation request. Updating the indexed documents changes the retrievable knowledge; this operation does not retrain the generation model.
Offline knowledge-base encoding stores chunk vectors in the vector database; online question answering searches those stored vectors and passes the retrieved chunks to generation.
A similarity metric defines how vectors are compared. Milvus calls its cosine metric COSINE, its inner-product metric IP, and its Euclidean metric L2. Cosine measures the angle between vectors: larger cosine values indicate closer alignment. Smaller L2 values indicate nearer vectors. Scores therefore need the metric’s interpretation; a larger number is not universally a better match.
Cosine similarity ranges from −1 to 1. A score of 0.8 is not an 80% probability that a passage answers correctly. Nor can you assume equal-dimensional vectors from unrelated embedding models represent the same space. See 6.1.3 for embedding-model selection and reranking, and 6.2.2 for similarity and index tuning.
The NVIDIA RAG Blueprint implements the separation with NeMo Retriever embeddings, a vector database (Elasticsearch by default, Milvus as an alternative), and a generation model. Its reference flow retrieves candidate content and joins relevant chunks into the generator’s context. Retrieval is an application stage; it need not involve an agent choosing tools.
Key distinctions and trade-offs
Retrieval versus generation: Search can return a useful passage while the generator misstates it. A fluent answer can hide a retrieval miss. Evaluate both stages separately.
More candidates versus more context: A wider search can find material a narrow search misses, but passing every candidate to generation increases input tokens and can include irrelevant text. Candidate retrieval and context selection are different decisions.
Semantic match versus exact match: Related wording helps with paraphrased questions; an exact product code or policy identifier can require a different retrieval strategy. See 6.1.2 for dense, sparse, and hybrid retrieval.
Architecture / decision scenario
An insurer’s employee assistant answers questions over 20,000 policy documents under a fixed served context limit. In an illustrative test, retrieving five chunks finds the required clause in 82 of 100 cases; retrieving twenty finds it in 94. Passing all twenty to generation, however, exceeds the team’s 12,000-token input allowance on long policies.
The tempting proposal is to keep only five because they fit. The retrieval owner tests the larger candidate set with the Blueprint’s optional NeMo Retriever Reranker to select context within the allowance; see 6.1.3. Reranking adds a model call per query and latency to the recall gain. The generation owner checks whether answers accurately use that selected text. Not every candidate belongs in the prompt.
Common exam traps
- Treating an embedding as the generated answer or as a copy of the original text.
- Reading similarity as factual confidence.
- Embedding questions with
input_type: "passage"on models that distinguish query and passage modes. - Assuming every metric ranks larger scores first.
- Equating RAG with retraining, or assuming all RAG systems are agentic.
What to remember
Encode documents and queries compatibly, interpret scores using the chosen metric, and distinguish candidate retrieval from the context sent to generation. RAG supplies source material; answer correctness still needs assessment.
Knowledge check
A team indexes its policy library with a NeMo Retriever Embedding NIM model that operates in passage and query modes, sending input_type: "passage" for every chunk. To keep the code simple, the question path also sends input_type: "passage", and retrieval accuracy is poor. Which change addresses the cause?
input_type: "query" to match the questionsinput_type: "query" and keep the passage indexpassage during indexing and query for questions, and the wrong mode reduces retrieval accuracy. Matching the question side to its documented mode removes the cause; re-indexing in query mode moves the mismatch to the documents, and a wider search or a retrained generator works around a retrieval defect without removing it.A retrieval service drops every hit scoring below 0.4 before it builds context. The team migrates its Milvus collection from the COSINE metric to L2 and keeps the same filter unchanged. What is the likely effect?
COSINE , a larger value indicates closer alignment; with L2 , a smaller value indicates greater similarity. A filter that keeps scores of at least 0.4 therefore rejects near vectors below that cutoff and retains farther ones above it, so each metric’s scores need their own interpretation and threshold.A product manager wants the assistant to label an answer “80% likely correct” whenever its top retrieved chunk has a cosine similarity of 0.8 to the question. What is wrong with that label?
An HR assistant uses RAG over the company’s policy documents. A revised leave policy replaces the current one tomorrow, and answers must reflect it from then on. Which change updates the retrievable knowledge to the revised policy?
References
- NVIDIA, NeMo Retriever Embedding NIM
- NVIDIA, Use the API (OpenAI) for NVIDIA NeMo Retriever Embedding NIM
- Milvus, Metric Types
- NVIDIA, NVIDIA RAG Blueprint