AWSAWS Certified Generative AI Developer - Professional2026-101.4 Design and implement vector store solutions
Balance vector relevance against retrieval cost
Completed by the knowledge check
Learning objective
Choose similarity metrics and index designs based on semantic relevance, query latency, and operating cost.
Core idea
A vector index implements a search contract, not a guarantee of useful evidence. The embedding model defines a representation; the distance metric ranks vectors within that representation; the index design determines how much work the search performs. You need evidence at all three layers before choosing the cheapest configuration that meets the application’s retrieval requirements.
Separate semantic relevance from nearest-neighbor recall. A search can recover the mathematically nearest vectors perfectly and still return passages that do not answer the question. Conversely, an approximate index can miss some nearest neighbors while retrieving enough useful evidence for an acceptable answer. Benchmark both outcomes. Embedding selection belongs to 1.5.3; this lesson holds that choice fixed while evaluating the search mechanism.
How it works
Match the distance to the representation
Amazon OpenSearch Service supports exact and approximate k-nearest-neighbor search. Its vector search documentation lists Euclidean distance, cosine similarity, and dot product. You set the metric per vector field with the space_type parameter of the knn_vector mapping, for example "space_type": "l2"; the auto-optimize console exposes the corresponding space types l2, cosinesimil, and innerproduct.
Euclidean distance measures separation between vectors. Cosine similarity compares their direction. Inner product combines direction and magnitude. For unit-normalized vectors, the three measures produce the same ranking; without normalization, magnitude can change the order. This is a mathematical property, not a promise that any embedding model supports interchangeable settings. Preserve the model’s documented normalization and similarity contract when comparing indexes.
Do not select cosine because a question contains the word “semantic.” Select a metric compatible with the embedding representation, then validate its retrieval behavior. Record the embedding model, vector dimensions, normalization, and space type together as an application configuration. Changing only the metric changes how stored vectors are interpreted; changing the model requires a compatible corpus rebuild, as discussed in 1.4.11.
Choose the work the index performs
For an OpenSearch k-NN index, index.knn enables the feature and a knn_vector mapping declares the vector field with its required dimension. The knn query supplies the query vector and k. Also set the top-level size: the AWS guide warns that omitting it can return k results per shard and segment rather than the intended total for the request. These parameters control result collection; they do not establish semantic quality.
Exact search provides a nearest-neighbor baseline for a fixed metric and eligible corpus. Approximate search uses structures such as Hierarchical Navigable Small World (HNSW) to reduce search work on large datasets. Approximation exchanges some search completeness for speed and resource efficiency. Measure that exchange on the corpus you actually serve, including the permission-filtered subsets described in 1.4.6.
The OpenSearch auto-optimize console accepts vector dimension, space type, desired recall, and latency requirements. Its recall example defines 0.95 as recovering, on average, 19 of the 20 true nearest document vectors. Use recommendations as candidates for validation, rather than treating that target as a measured production result. A target tells the optimizer what you want; a benchmark tells you whether you obtained it.
Include memory and disk behavior
AWS describes disk-optimized search for OpenSearch Serverless vector collections as a two-phase design. In in_memory mode, the HNSW graph resides in RAM. In on_disk mode, compressed vectors support candidate generation, and full-precision vectors are read from disk for final rescoring. AWS documents the default on_disk mode as 32x binary quantization, reducing memory requirements by 97% versus in-memory mode, with P90 latency in the 100–200 ms range. Treat those as documented characteristics to verify, not guarantees for your corpus. Quantization adds another approximation boundary and disk work.
Compression is a mapping decision. In NextGen Serverless vector collections, every index is created with 32x compression by default, and the compression_level parameter of the knn_vector mapping overrides it, for example "compression_level": "1x". Serverless bills compute in OpenSearch Compute Units (OCUs), separately from storage, so the memory reduction acts on the OCUs a vector workload needs. Do not present this Serverless guidance as a universal setting for every backend in 1.4.1.
For managed OpenSearch domains, monitor the CloudWatch KNNGraphMemoryUsage metric on each data node against knn.memory.circuit_breaker.limit and available RAM. The AWS k-NN guide explains that the service uses half the instance RAM for Java heap, up to 32 GiB, and k-NN defaults to 50% of the remaining half. Its 32-GiB instance example therefore permits 8 GiB of graphs. Disk capacity alone does not prove that an in-memory vector workload fits.
Key distinctions and trade-offs
Compare candidates using the following evidence:
| Choice | Evidence that decides it | Cost to include |
|---|---|---|
| Distance metric | Compatibility and relevance of ranked evidence | A rebuild or retest when the representation changes |
| Exact versus approximate search | Nearest-neighbor recall and acceptable passage retrieval | Search computation and tail latency |
| Memory versus disk-oriented design | Warm and cold query latency at required recall | Memory, disk reads, and rescoring work |
| Returned candidate count | Useful evidence after filtering and selection | Retrieval work and downstream context processing |
Increasing the number of candidates can improve coverage, but it also sends more work to later stages. Include retrieval and downstream generation in cost per accepted answer. Keep this application measurement separate from AWS billing units; a low infrastructure bill can still correspond to expensive failed answers.
Test concurrent reads while ingestion is active. Report p95 or p99 latency alongside averages, and include small authorized subsets, rare terminology, and unanswerable questions. A benchmark dominated by easy queries can hide precisely the misses that cause unsupported answers.
Architecture / decision scenario
A support assistant searches a large manual collection. The team fixes its embedding contract, compares compatible space types, and builds an exact baseline for representative authorized queries. An HNSW candidate meets the latency target but loses important troubleshooting passages; a higher-quality candidate restores those passages at increased memory cost.
The team then evaluates an OpenSearch Serverless disk-optimized candidate. It lowers memory demand but adds rescoring latency. Because the product permits that latency and the reviewed answers still pass, the team selects it for the large collection. A separate latency-sensitive collection retains the configuration that meets its tighter target. Neither decision relies on the same recall number alone.
Common exam traps
- Equating vector proximity with factual support for a generated claim.
- Choosing a distance metric without checking normalization and model compatibility.
- Assuming approximate search returns every true nearest neighbor.
- Omitting
sizewhile expectingkto limit the complete multi-shard response. - Sizing an in-memory index from disk capacity while ignoring graph memory.
- Applying Serverless disk-optimization guidance to an unrelated store without verification.
- Assuming a NextGen Serverless index stores full-precision vectors in memory when no
compression_levelis set.
What to remember
Choose the representation-compatible metric first, then benchmark the index design. Evaluate semantic evidence quality separately from nearest-neighbor recall. Use index.knn, knn_vector, dimension, k, and size to identify the OpenSearch interface, and use memory and latency measurements to decide whether the design is economical. Compression is a candidate to test, not an automatic permission to accept worse retrieval.
Knowledge check
An HNSW index recovers 19 of the 20 true nearest neighbors on average, measured against an exact baseline. Reviewers still find that many returned passages do not answer the policy questions. What does this show?
A managed OpenSearch domain runs on data nodes with 32 GiB of RAM and default k-NN settings. The team estimates 12 GiB of HNSW graphs per node, and each node has plenty of free disk. Will the graphs fit in memory?
KNNGraphMemoryUsage against knn.memory.circuit_breaker.limit .Two index candidates pass the same relevance gates. An OpenSearch Serverless on_disk candidate cuts memory sharply but adds rescoring latency. The product’s latency target is 300 ms at P90. What should decide whether to choose it?
An embedding model outputs vectors that are not normalized, and the team wants ranking to depend on direction only, not on vector length. Which space_type should it evaluate first?
l2 , because Euclidean distance ignores vector lengthcosinesimil , because cosine similarity compares directioninnerproduct , because inner product compares direction onlyAn OpenSearch index has five shards. A knn query sets k to 10 and returns far more than 10 hits. What should the team change?
k , so that the five shards together return ten hitsk applies oncesize to 10, alongside the knn queryk , so that the approximate search can stop soonersize , the query returns k results for each shard and segment rather than k for the whole request. Setting size limits the total, leaving k to control the neighbor search itself.