AWSAWS Certified Generative AI Developer - Professional2026-101.4 Design and implement vector store solutions
Scale retrieval without multiplying search work
Completed by the knowledge check
Learning objective
Scale OpenSearch retrieval through shards, replicas, multi-index strategies, and hierarchical indexing.
Core idea
Scaling retrieval means controlling the work each request creates while distributing storage and concurrency. Primary shards partition an index. Replicas copy those partitions. Multiple indexes divide the corpus into separately managed search spaces. A hierarchical retrieval design selects a coarse search space before searching its detailed evidence. These choices solve different problems and can work against one another.
The shard and replica discussion here applies to Amazon OpenSearch Service managed domains. In OpenSearch Serverless, AWS states that “the number of shards, number of intervals, and refresh interval are not modifiable and are handled by OpenSearch Serverless”; a collection scales through OpenSearch Compute Units (OCUs), provisioned separately for search and indexing, rather than through number_of_shards. Similarity and vector-memory choices belong to 1.4.7; the question here is how those indexes are arranged and how much of them each request touches.
How it works
Distribute partitions with primary shards
The number_of_shards index setting defines primary partitions. OpenSearch Service distributes them across the domain’s data nodes. If you set nothing, OpenSearch Service divides each index into five primary shards with one replica each, ten shards in total, unlike open-source OpenSearch’s default of one primary and one replica. AWS advises deciding the primary count before indexing because changing it for an existing index is not straightforward. An index template can set number_of_shards, number_of_replicas, and refresh_interval consistently for matching indexes.
Begin with measured indexed size and anticipated growth, rather than a document count alone. AWS’s general guidance is 10–30 GiB per shard for search-latency-sensitive workloads and 30–50 GiB for write-heavy workloads. Those ranges are starting points, not vector-search guarantees: also apply the graph-memory budget from 1.4.7. A shard can fit on disk while its vector workload exhausts memory.
Too few large shards restrict distribution and complicate recovery. Too many small shards consume CPU and memory for coordination and maintenance; AWS’s ceiling is no more than 25 shards per GiB of Java heap on a node, and its initial scale point is 1.5 vCPU per shard. AWS recommends considering shard count relative to data-node count, but gives shard size priority: a 5-GiB corpus should still use one primary shard. Do not create dozens of tiny partitions solely to make the count divisible by the number of nodes.
Add copies for concurrent reads and resilience
The number_of_replicas setting adds copies of each primary shard. AWS states that primary and replica copies reside on different data nodes, and recommends at least one replica. Additional replicas increase redundancy and read capacity, but also increase storage and the work required to maintain copies during indexing.
For the ordinary domain search behavior described in AWS’s operational best practices, a coordinator sends a search to either a primary or replica copy of every primary partition involved. An index with five primaries and one replica has ten shard copies, but a search touches five partitions, not ten. Adding replicas gives concurrent requests more copies to use; it does not halve the amount of corpus one request must search.
Scope that rule to the storage architecture. AWS’s multi-tier documentation specifies different warm-index behavior: searches go to the primary, and a replica serves reads when the primary is down. Do not infer equal steady-state read scaling from every replica in every tier. Choose resilience and capacity from the documented behavior of the actual deployment.
Split indexes by a useful boundary
Multiple indexes can separate embedding versions, domains, tenant groups, or lifecycle classes. The boundary earns its cost when the application can search a smaller eligible subset or manage a distinct rebuild. Searching every index on every request preserves fan-out and adds operational objects; splitting alone does not reduce latency.
Amazon OpenSearch Ingestion provides concrete ingestion mechanisms for such layouts. The OpenSearch sink’s index value can use fields from incoming events to generate index names. Its document_id option preserves an event-derived document identifier. Its routing_field option sets the indexed document’s _routing value. These are placement mechanisms, not authorization policies. Permission checks and trusted filtering still follow 1.4.6.
Treat the index family and routing convention as a shared contract for writes, reads, updates, and deletions. If you route by a source field, changing that field requires reconciling the old placement as well as writing the new one. Budget total shard copies across the entire index family, including temporarily retained rebuild generations.
Build a hierarchy before detailed retrieval
Hierarchical indexing in this design is an application pattern, not a promise of a single AWS switch. Maintain a coarse OpenSearch index of domain or document summaries and detailed indexes of evidence chunks. A first _search identifies candidate parents; a second search targets their detail indexes or applies their identifiers as trusted constraints. Keep the parent-to-child relationship in your own metadata schema.
The benefit is a smaller detailed search space. The risk is a first-stage miss that makes relevant evidence unreachable. Measure parent-selection recall and end-to-end evidence recall separately. An ambiguous question might require searching several branches or using a bounded broader fallback. Parent summaries alone should not replace the detailed source evidence.
The detailed search reaches only the parents the coarse search selects, widened for an ambiguous question.
This is different from Knowledge Bases hierarchical chunking in 1.5.1. Chunking governs how a document becomes retrievable pieces. The hierarchy here governs which corpus partitions the application searches. Neither a parent selector nor a shard boundary can repair missing source content.
Key distinctions and trade-offs
Use the bottleneck to choose the control:
| Observed bottleneck | Candidate control | Failure to watch for |
|---|---|---|
| Storage concentrated in a few partitions | Primary-shard layout and balanced placement | Excessively small shards |
| Concurrent reads saturate available copies | Replica count and data-node capacity | Higher indexing and storage cost |
| Requests search unrelated domains | Explicit multi-index selection | Missing a relevant index |
| Detailed corpus search is too broad | Coarse-to-fine hierarchy | First-stage recall loss |
Compare layouts under active ingestion, peak concurrency, and a degraded-node scenario. Count the partitions each request searches; a faster isolated query does not establish better throughput. Include rebuild overlap in storage planning and test whether fallback queries erase the hierarchy’s expected savings.
Architecture / decision scenario
An engineering assistant serves three product families. Its single large index suffers concurrent read pressure, while most questions name one family. The team sizes primary shards from measured disk and graph memory, then adds replicas for the required read concurrency and resilience.
It separates product-family indexes and uses event-derived index names and stable document_id values during ingestion. A coarse summary index selects families for ambiguous questions, followed by detailed chunk searches. Evaluation includes cross-family questions; when the coarse selector lacks enough evidence, the application searches a bounded set of additional families. The team rejects a design that creates one tiny index per manual because total shard overhead exceeds its routing benefit.
Common exam traps
- Treating replicas as additional partitions of unique data.
- Assuming more shards always reduce latency.
- Expecting an index created without settings to have one primary shard, or tuning
number_of_shardson a Serverless collection. - Splitting indexes while continuing to search all of them for every request.
- Confusing
_routingplacement with tenant authorization. - Calling hierarchical chunking and corpus-level hierarchical retrieval the same mechanism.
- Applying ordinary replica read behavior to warm tiers without checking their documentation.
What to remember
Primary shards distribute data; replicas add copies; multi-index selection narrows eligible search spaces; a hierarchy adds a selection stage. Use each control for its own bottleneck. Measure fan-out, memory, concurrent throughput, and both stages of recall before accepting a scaling plan.
Knowledge check
A managed OpenSearch domain holds one vector index with five primary shards and one replica each. Single queries meet their latency target, but throughput saturates when many users search at once. Which change targets this bottleneck?
A team creates a vector index for a 5 GiB corpus on a managed domain with six data nodes, leaving the shard settings at their defaults. How many primary shards should the index have?
A team splits its corpus into one index per product family, but every request still searches all of the indexes. Latency has not improved. What change lets the split pay off?
k in each index query, so that every search returns lessA coarse summary index chooses product families, and then detailed chunk indexes are searched only within those families. For some questions, the relevant family is never selected. Which measurement exposes this loss?
An OpenSearch Ingestion pipeline sets routing_field to tenant_id, so each tenant’s documents share a routing value. A developer proposes dropping the tenant filter from queries. What is the correct assessment?
_routing restricts each search to the caller’s documentsdocument_id fieldrouting_field sets the indexed document’s _routing value, which governs placement. It is not a permission policy: trusted caller identity, a mandatory tenant filter and document checks still decide what a request may retrieve.