NVIDIANVIDIA-Certified Professional: Agentic AI2026-106.1 Implement retrieval pipelines (RAG, embedded search, hybrid approaches).
Select embeddings before tightening reranker precision
Completed by the knowledge check
Learning objective
Select embedding models and add reranking for precision.
Core idea
Embedding retrieval creates a candidate pool; reranking chooses the most useful passages within it. Choose an embedding model that finds the right material for your language, domain, and input modality, then decide whether a reranker improves the context enough to justify another inference stage.
A good reranker cannot select an absent passage. A strong embedding model can still return several plausible but irrelevant neighbors. Evaluate those failures separately: candidate coverage belongs to retrieval, while ordering and retention belong to reranking. See 3.1.3 for the quality metrics; this lesson connects the diagnosis to the model interfaces and serving controls.
How it works
Match the embedding interface to the corpus
NVIDIA NeMo Retriever Embedding NIM exposes POST /v1/embeddings. For asymmetric models such as nvidia/nemotron-3-embed-1b, use input_type="passage" when indexing documents and input_type="query" for questions; see 0.3. The documentation warns that incorrect modes reduce retrieval accuracy. A successful response with a vector is therefore insufficient verification of the ingestion path.
Choose candidate models using representative questions and passages from the corpus, including the languages and technical vocabulary the assistant must handle. Test cross-language questions separately from questions whose source language matches. An overall score can hide a model that serves the dominant language well and misses another required language.
Input modality is also model-specific. The Embedding NIM API guide says nvidia/nemotron-3-embed-1b accepts text only. For nvidia/llama-nemotron-embed-vl-1b-v2, image documents use input_type="passage"; images supplied in query mode are rejected. Choose a text pipeline when extracted text carries the answer, and inspect multimodal retrieval when layout or visual content matters; see 2.2.1 for that routing decision.
In the Blueprint, select models with APP_EMBEDDINGS_MODELNAME and APP_RANKING_MODELNAME. The documented defaults are nvidia/llama-nemotron-embed-vl-1b-v2 and nvidia/llama-nemotron-rerank-1b-v2, respectively. Record model identity, mode, dimensions, and preprocessing with the collection. Ingestion and retrieval must use the same embedding model and dimensions; changing either requires reingestion. The VL embedder’s vectors and the text-only nvidia/llama-nemotron-embed-1b-v2 vectors are not directly comparable despite both having 2,048 dimensions.
Budget representation size without assuming support
The dimensions parameter is model-specific. The API guide states that nvidia/nemotron-3-embed-1b supports only its native 2,048-dimensional output: omit dimensions or set it to 2048. Reduced dimensions are not supported for that model. Do not take the presence of a parameter in a shared API as proof that each model implements every value.
For nvidia/nemotron-3-embed-1b, embedding_type supports float, int8, uint8, binary, and ubinary. Float is the default. The int8 and uint8 forms use one byte per dimension, a 4× reduction from float32; binary forms use one bit, a 32× reduction. One million 2,048-dimensional vectors therefore require about 8.19 GB as float32 or 2.05 GB as int8 before indexes, metadata, and replicas.
Binary forms require Hamming distance rather than cosine similarity or dot product. Not every vector database supports compressed types, and embedding_type cannot be combined with dimensions. Check database support and retrieval quality before choosing compression; see 6.2 for index and capacity decisions.
Every embedding model also has a token-length boundary. The truncate field controls overlength handling with START, END, or NONE. Check tokenized passage length against the served profile. Truncation can produce a valid embedding while discarding the sentence needed to answer. Repair systematic overlength inputs through chunking, as described in 6.1.1, rather than accepting silent loss as successful ingestion.
Rerank a bounded candidate pool
NeMo Retriever Reranking NIM accepts POST /v1/ranking with a query and a list of passages. It scores query-passage content rather than reusing the vector store’s similarity scores. Its response contains rankings entries with index and logit, ordered by descending logit. Use each returned index to recover the corresponding input passage and its source metadata.
A logit is an unnormalized model prediction, not a calibrated probability that a passage is correct. Preserve the distinction when setting thresholds or comparing models. Do not interpret a logit of zero as zero confidence or transfer a cutoff between scoring systems without checking its meaning.
The documented maximum is 512 passages per reranking call. The model’s sequence limit applies separately to each query-passage pair after tokenization. With truncate="NONE", an overlength pair returns an error. With truncate="END", the service removes tokens from the passage’s end until the pair fits. If the query itself reaches the limit, the whole passage can disappear. Shorten an unnecessarily long query or split oversized passages instead of assuming truncation preserves relevance.
Separate candidate breadth from answer context
In NVIDIA RAG Blueprint, vdb_top_k controls retrieved candidates and reranker_top_k controls retained chunks. The settings guide lists starting values of 100 and 10, respectively, and ENABLE_RERANKER=true enables the model stage. These are separate controls: raising candidate breadth increases reranker work, while raising retained context increases the material sent for generation.
RERANKER_SCORE_THRESHOLD defaults to 0.0 and requires ENABLE_RERANKER=true. The settings guide recommends 0.3–0.5 to balance quality and coverage, but warns that a high threshold can remove too many chunks and leave no response. Tune it against the Blueprint’s score behavior and labeled cases; do not substitute a cutoff copied from the raw NIM logit output.
Use retrieval_time_ms and context_reranker_time_ms to locate latency growth. If useful candidates reach reranking but disappear from the final context, inspect the threshold and reranker_top_k. If the candidates are already irrelevant, change the embedding or retrieval path before buying a larger reranker.
Key distinctions and trade-offs
Select the stage that repairs the observed defect:
| Decision | Deciding condition | Mechanism | Cost |
|---|---|---|---|
| Embedding model | Relevant passages are missing from candidates | /v1/embeddings, model, input_type | Passage re-embedding and replacement collection |
| Representation size | Vector storage exceeds budget | Supported dimensions or embedding_type | Model-specific support and possible quality loss |
| Reranker | Candidates contain answers but their ordering is weak | /v1/ranking, query, passages | Pairwise inference, added latency, and hardware for self-hosting |
| Retained context | Useful reranked passages are excluded or too much context is sent | reranker_top_k, RERANKER_SCORE_THRESHOLD | Coverage versus generation tokens and latency |
Architecture / decision scenario
A procurement assistant searches supplier documents in two required languages. It must retain answer-bearing passages in at least 95% of cases for each language and keep retrieval plus reranking below 500 ms at the 95th percentile. In an illustrative trial, embedding candidate A retrieves the needed passages in 98% and 82% of the two language slices; candidate B reaches 97% and 96%.
The retrieval owner chooses B. The tempting alternative—keep A and add a stronger reranker—cannot repair its missing second-language candidates. Both passage collections are built with their respective model and passage-mode settings so the comparison does not mix vector spaces.
For B, reranking 80 candidates and retaining eight raises answer-context precision from 61% to 86%, with a measured 430 ms combined latency. Increasing the candidate pool to 200 reaches 87% precision but takes 690 ms and fails the budget. The serving owner selects the 80-to-eight path and tests threshold-induced empty results before release.
Common exam traps
- Embedding queries and documents in the same mode for an asymmetric model.
- Assuming equal vector dimensions make two embedding models compatible.
- Treating optional
dimensionsas universally supporting smaller outputs. - Expecting reranking to recover passages outside the candidate pool.
- Reading raw logits as probabilities or losing passage identity after sorting.
- Maximizing both top-k settings without accounting for their different costs.
What to remember
Select embeddings for candidate coverage and supported inputs, then use reranking to improve precision within a bounded pool. Keep model identity, query/passage modes, dimensions, token limits, and returned indexes explicit. Measure the extra inference stage against the complete retrieval budget.
Knowledge check
A new ingestion script indexes documents with nvidia/nemotron-3-embed-1b and sends input_type="query" on every call, the same value the question path uses. Every call returns a vector, but retrieval accuracy has dropped sharply since the script went live. Which fix addresses the cause?
passage modepassage mode and keep query for questionsdimensions for both pathsinput_type="passage" and questions with query , and NVIDIA warns that the wrong mode causes large accuracy drops. Making both paths use one mode repeats the mistake on the other side, and a reranker cannot fix candidates the mismatched vectors never surfaced. A returned vector proves only that the call succeeded.A procurement assistant must retrieve the answer-bearing passage in at least 95% of cases for each of its two required languages. In a trial, embedding model A reaches 99% in the first language and 79% in the second; model B reaches 97% and 96%. Model A’s index already exists. What should the retrieval owner do?
A team stores one million 2,048-dimensional nvidia/nemotron-3-embed-1b vectors and must cut vector storage about fourfold. Their vector database supports int8 vectors, and search must keep using inner product. Which request change fits?
dimensions=512 and keep the default float outputdimensions=1024 together with embedding_type="int8"embedding_type="binary" and keep inner-product searchembedding_type="int8" at the native 2,048 dimensionsint8 uses one byte per dimension instead of four, a fourfold reduction. The model supports only its native 2,048-dimensional output, so reduced dimensions values are not supported, and dimensions cannot be combined with embedding_type in any case. Binary output compresses 32-fold but requires Hamming distance rather than inner product.A Blueprint assistant retrieves 100 candidates. The passage needed for a multi-part answer is reranked ninth, passes the score threshold, but is absent from the eight chunks sent to generation. The context budget permits ten chunks. Which change directly addresses the loss while keeping retrieval and reranking models fixed?
reranker_top_k from eight to tenvdb_top_k from 100 to 200RERANKER_SCORE_THRESHOLD for retained chunksRERANKER_SCORE_THRESHOLD for retained chunksreranker_top_k controls how many reranked chunks reach generation. The needed passage is already retrieved and clears the threshold, so increasing the retained count includes it. Fetching more candidates changes the input pool, while changing the threshold addresses a different reason for exclusion.