NVIDIANVIDIA-Certified Professional: Agentic AI2026-106.2 Configure and optimize vector databases for fast retrieval.
Tune retrieval against filtered exact results
Completed by the knowledge check
Learning objective
Tune similarity metrics, index parameters, metadata filters, and GPU acceleration for recall–latency trade-offs.
Core idea
A retrieval operating point combines the embedding metric, search effort, eligible records, and execution hardware. Tuning one dimension while silently changing another produces an invalid comparison. Freeze the corpus and queries, define the eligible set for each request, and measure ANN recall against exact neighbors under that same definition.
Index selection belongs to 6.2.1. Here you decide how much work the selected index performs, whether its metric matches the vectors, and whether filtering or GPU execution changes the result or cost. Record the configuration and measurements together so the deployment owner can reproduce the chosen point.
How it works
Preserve the meaning of similarity
See 0.3 for the L2, IP, and COSINE similarity definitions; here, set Milvus’s metric_type to match the embedding model’s retrieval convention. Normalizing both stored and query vectors to unit length makes their inner product equal cosine similarity. Model choice belongs to 6.1.3.
The Milvus GPU-index overview excludes direct COSINE support and documents normalized vectors with IP as the substitute. Apply normalization consistently on both ingestion and querying paths. Otherwise, a migration from CPU cosine search to GPU inner product changes the ranking problem while appearing to change only hardware. Recompute the exact reference if you intentionally change the representation or metric.
Distinguish construction from search effort
For IVF_FLAT, fix the built nlist and sweep nprobe in search_params.params. Milvus permits nprobe values from 1 through nlist. More probes examine more clusters and usually recover more neighbors at greater query cost. If no probe count meets the target, compare another construction rather than pretending query tuning repairs every clustering choice.
For HNSW, sweep ef with the same M and efConstruction. The latter two affect the graph built in advance; ef changes query breadth. Milvus recommends an ef range from K to 10K, where K is the requested neighbor count. Treat this as a documented tuning recommendation, not a guarantee of recall or a universal maximum.
For GPU CAGRA, itopk_size retains intermediate candidates and should be at least the final search limit. Increasing search_width explores more graph entry points. Both can improve recall at a performance cost. Keep graph construction fixed while sweeping these search settings; otherwise, the source of an improvement becomes ambiguous.
Filter within the retrieval operation
Milvus standard filtered search takes a filter expression, identifies matching entities, and performs ANN retrieval within that set. Use this for eligible document metadata, such as authenticated tenant scope and document status. Construct authorization predicates in trusted application code; a user-supplied natural-language request must not decide whether that predicate exists.
NVIDIA RAG Blueprint’s /v1/search and /v1/generate payloads expose filter_expr; for Milvus, use a boolean string over fields such as content_metadata["tenant_id"]. The enable_filter_generator option, disabled by default and configurable through ENABLE_FILTER_GENERATOR, generates filters from natural language. It can also rewrite a supplied filter_expr. NVIDIA documents an empty-filter fallback when the LLM is unavailable, so a tenant predicate cannot depend on successful generation. Keep enable_filter_generator=false on this authorization-sensitive path and supply the trusted predicate in filter_expr; enforce the collection privileges described in 6.2.3 as well.
Highly complex scalar expressions can dominate standard-filter latency. Milvus also documents iterative filtering through search_params with hints: "iterative_filter". It retrieves candidates in iterations and applies scalar checks until the requested top K is reached. This can reduce scalar-filter work, but sequential checking can become slow when many candidates need inspection. Benchmark the actual expression and selectivity rather than assuming iterative filtering always wins.
This database operation differs from fetching a small global top K and removing forbidden results in application code. That shortcut can leave fewer eligible neighbors than requested and overlook better eligible results outside the global candidate set. If only six eligible records exist for K=10, however, returning six is correct; increasing search breadth cannot manufacture four more records.
Standard filtered search identifies matching entities before ANN retrieval, while the application shortcut removes forbidden results after a global top K and can leave fewer eligible neighbors than requested.
Account for GPU execution separately
Milvus loads collection data into GPU memory before GPU searches. From Milvus 2.4.1 onward, gpu.initMemSize and gpu.maxMemSize configure a pool for temporary search data; the GPU-index overview recommends values of 2048 and 4096, respectively. This pool does not determine whether the collection fits: budget device memory for loaded data, approximately 1.8 times the raw vectors for GPU_CAGRA, plus temporary search data and other allocations. The page warns that incomplete loading out-of-memory protection can let excessive data crash a QueryNode. Increasing search effort does not repair that capacity deficit.
NVIDIA RAG Blueprint exposes APP_VECTORSTORE_ENABLEGPUINDEX and APP_VECTORSTORE_ENABLEGPUSEARCH separately for its optional Milvus backend. GPU construction with CPU search activates adapt_for_cpu and requires query ef, rather than the GPU traversal settings alone. Keep a GPU-capable Milvus image for construction even when queries run on CPU. Benchmark the serving path you actually deploy. A build-time GPU optimization is not a measured GPU query-speed improvement.
Key distinctions and trade-offs
Compare the controls by what they change:
| Control | Deciding condition | Mechanism | Cost or failure risk |
|---|---|---|---|
| Metric | Match the embedding’s similarity convention | metric_type, normalization | Changed ranking if representation differs |
| IVF breadth | Required neighbors fall outside visited clusters | nprobe at fixed nlist | More comparisons per query |
| HNSW breadth | Graph traversal misses exact neighbors | ef at fixed construction | Longer traversal |
| Metadata filtering | Only eligible records belong in results | filter; optional iterative_filter hint | Scalar evaluation or repeated candidate checks |
| GPU traversal | Device serving improves the accepted workload | itopk_size, search_width, GPU-search setting | Device memory and query-serving allocation |
Record recall@K as the overlap with exact top-K IDs divided by K when at least K eligible records exist. Define a separate convention for smaller eligible sets. Measure latency percentiles at the intended concurrency and batch size; an idle single-query result cannot establish a loaded service target.
Architecture / decision scenario
A legal research assistant searches tenant-scoped case documents. Its project targets are recall@10 of 0.98 and retrieval p95 below 70 ms at 40 concurrent requests. Each query must include the authenticated firm’s tenant_id and an active-document condition. These are scenario requirements, not documented service limits.
The retrieval engineer compares normalized IP vectors with a filtered exact reference before evaluating a GPU migration. In a hypothetical HNSW trial with ef=80, standard filtering takes 92 ms for complex metadata expressions; iterative filtering takes 54 ms and retains 0.985 recall with the same predicate. On another query slice where only 0.1% of records qualify, iterative filtering takes 130 ms. Choose from the measured slice behavior rather than replacing all standard searches globally.
The tempting alternative is to drop the filter, retrieve ten global hits, and remove other firms’ documents afterward. The security owner rejects it: it changes the eligible search problem and can starve the answer of authorized passages. The platform owner also rejects a faster GPU CAGRA candidate needing 20 GiB of loaded data against 18 GiB of GPU memory available to Milvus. Those figures describe loaded data and device capacity; adjusting the temporary search pool does not close that gap. Tune search breadth within each supported serving configuration, then accept only a point meeting quality, filter correctness, memory, and latency requirements together.
Common exam traps
- Relying on
enable_filter_generatorto add or preserve the tenant predicate. - Replacing cosine with unnormalized inner product during a GPU migration.
- Comparing ANN results with an unfiltered exact reference for a filtered query.
- Raising HNSW K above
ef, or CAGRAlimitaboveitopk_size, without increasing search breadth. - Calling iterative filtering universally faster, regardless of predicate selectivity.
- Applying GPU search parameters to a CPU-adapted CAGRA index without
ef.
What to remember
Fix the metric and eligible set before tuning search effort. Sweep query parameters separately from construction parameters and compare against matching exact results. Filters and GPU settings belong in the measured configuration; neither security requirements nor device memory limits disappear when an index becomes faster.
Knowledge check
A team moves a Milvus collection of unnormalized vectors searched with COSINE on CPU to a GPU index and wants the ranking problem to stay unchanged. Milvus GPU indexes do not support COSINE. How should they configure the GPU index?
IP on the existing, unnormalized vectorsL2 on the existing, unnormalized vectorsCOSINE and switch to GPU_BRUTE_FORCEIPIP as the GPU substitute; normalization must apply on both the ingestion and the query path. Unnormalized IP or L2 changes which neighbors rank first, and the COSINE restriction covers GPU indexes in general.A multi-tenant legal assistant calls the NVIDIA RAG Blueprint’s /v1/search on Milvus. Each request must be restricted to the authenticated firm’s tenant_id, which the application already knows from the session. How should the request carry that restriction?
filter_expr and set enable_filter_generator=falseenable_filter_generator=true and let the user’s question name the firmfilter_expr and set enable_filter_generator=truefilter_expr makes Milvus search only eligible records. The filter generator can rewrite a supplied filter and falls back to an empty filter when its LLM is unavailable, so an authorization predicate must not depend on it. Filtering a global top ten afterwards can leave fewer authorized passages than requested and miss better eligible ones.An HNSW search with a complex tenant-and-status filter must stay under 70 ms at p95. On a broad query slice, standard filtering takes 92 ms and iterative filtering 54 ms at equal recall. On a slice where 0.1% of records qualify, standard filtering takes 40 ms and iterative filtering 130 ms. Which configuration should the team adopt?
An HNSW index built with M=16 and efConstruction=200 reaches recall@10 of 0.91 at ef=16, and the target is 0.97. The team cannot rebuild the index this week. Which change should they test?
efConstruction to 400M from 16 to 32ef from 16 to 64ef is the query-time search breadth, so raising it can recover more true neighbors at the cost of latency without touching the graph. M and efConstruction shape the graph during construction and take effect only through a rebuild, and asking for more neighbors at the same breadth changes the request rather than the search effort.References
- Milvus, Metric Types
- Milvus, IVF_FLAT
- Milvus, HNSW
- Milvus, GPU_CAGRA
- Milvus, Filtered Search
- Milvus, GPU Index Overview
- NVIDIA, Advanced Metadata Filtering with Natural Language Generation
- NVIDIA, Milvus Configuration for NVIDIA RAG Blueprint