AWSAWS Certified Generative AI Developer - Professional2026-104.2 Optimize application performance

Tune retrieval speed behind relevance gates

Completed by the knowledge check

Learning objective

Improve retrieval speed through index tuning, query preprocessing, and hybrid scoring without losing relevance.

For an architect optimizing a retrieval-augmented generation application, the outcome is a controlled release decision that improves speed without dropping the evidence needed for a correct answer. See 1.4.7 and 1.4.8 for index design, 1.5.7 for hybrid retrieval, and 1.5.9 for query transformations.

Core idea

A retrieval pipeline can be made faster by doing less work, but less work can remove the only passage that supports the answer. Optimize latency subject to a relevance gate, not independently of it.

Separate query preparation, embedding, vector or lexical search, score combination, and evidence selection. A faster vector query does not prove the whole retrieval stage is faster. A faster retrieval stage does not prove the generated answer still has adequate evidence.

Start with a fixed corpus revision and labeled query set. Retain the baseline’s authorized result set, latency distribution, and evidence-coverage measurements. Change one layer at a time so you can identify which setting produces the trade-off.

How it works

Tune the index and candidate workload

For a custom Amazon OpenSearch Service vector index, the knn query’s k controls the requested neighbors. AWS documents a maximum k of 10,000. The request’s size also matters: without it, you can receive k results for each shard and segment rather than k results for the whole query.

Treat k and size as different controls. The first affects the neighbor request; the second bounds returned hits for the overall search. Reducing returned hits alone is not proof that all internal search work decreases. Measure the vector-call duration, returned payload, and subsequent evidence-processing time separately.

AWS warns that combining knn with other clauses can yield fewer than k results. Its post_filter example removes a retrieved result. This matters when evaluating speed: a smaller returned set might reflect filtering after candidate selection, not a better index or better relevance. Permission filtering remains a correctness boundary; see 1.4.6.

Check KNNGraphMemoryUsage on each data node against knn.memory.circuit_breaker.limit and available memory. AWS documents that graph memory pressure can harm performance. A search adjustment that looks effective on an underloaded index can fail under the corpus and concurrent demand you actually serve.

Index topology also affects resource use. AWS gives a general shard-size guideline of 10–30 GiB when search latency is a key objective. That is a sizing guideline, not a vector-search service-level guarantee. Too many small shards consume CPU and memory; too-large shards complicate recovery. Use the topology reasoning in 1.4.8 instead of assuming additional shards always make one query faster.

For Bedrock Knowledge Bases, retrievalConfiguration.vectorSearchConfiguration.numberOfResults requests source chunks, with a documented range of 1–100. Do not confuse this managed retrieval setting with OpenSearch k or size. Tune it as a candidate-budget control and verify that the returned evidence still contains the required sources.

Bound query preprocessing

Query preprocessing can remove irrelevant boilerplate, preserve identifiers in a canonical form, or split a multi-part question. Its latency belongs in the retrieval budget. A rewriting model that saves a small amount of vector-search time can still increase total response delay.

Keep the original query as the reference for intent. Preserve product codes, error identifiers, dates, exclusions, and permission constraints that determine the evidence. An application transformation that strips an exact identifier might improve a semantic match while destroying the useful lexical match.

Use a minimal deterministic transformation when its rule is sufficient. Introduce model-based rewriting only when measured retrieval coverage justifies its additional invocation. Validate the complete transformed query before submitting retrieval. Incremental rewrite output does not help a retrieval consumer that needs the entire validated query.

For managed query decomposition choices and their API contract, see 1.5.9.

Decomposition can improve evidence coverage for multi-part questions but introduces additional retrieval work. Do not advertise it as an automatic speed optimization. Benchmark the complete transformation-and-search path and retain a direct-query baseline.

Combine scores only on supported paths

Hybrid is available only on supported stores with a filterable text field; see 1.5.7 for the search settings and backend scope.

In a custom OpenSearch Serverless hybrid implementation, AWS documents a hybrid query and a normalization search pipeline processor. Supported normalization techniques include min_max and l2; supported combination techniques include arithmetic_mean, geometric_mean, and harmonic_mean.

Use normalization and combination to construct a consistent ranking from different score scales. Raw lexical and vector scores are not interchangeable measurements of correctness. Select the combination using labeled relevance outcomes, not whichever produces the largest numerical scores.

Hybrid search processes two retrieval signals and then normalizes and combines their scores in a search pipeline. Budget for both retrieval paths and the score-processing step rather than timing only vector search. Depending on execution overlap, added work can increase resource use without adding the same amount of wall-clock delay; measure both under representative concurrency.

Its benefit can be better evidence at a smaller acceptable candidate budget, rather than a cheaper individual query. Compare normalized hybrid ranking against the semantic baseline at equivalent evidence-coverage targets. Prove the complete latency and relevance benefit before shrinking the budget.

Key distinctions and trade-offs

Index speed versus pipeline speed: Query transformation, embeddings, payload transfer, and evidence assembly can dominate a faster index.

Candidate count versus relevance: A smaller result set saves downstream work only if the required evidence survives.

Query normalization versus intent change: Formatting a product identifier consistently differs from replacing it with a broad category.

Hybrid relevance versus latency: Combining lexical and semantic signals can improve ranking while adding execution work.

Use recall of required evidence and ranked relevance alongside retrieval p50 and p95. Keep permission outcomes as a separate mandatory gate. A relevance score cannot authorize a document, and latency improvement cannot excuse losing the only permitted supporting passage.

Architecture / decision scenario

A troubleshooting assistant answers natural-language questions mixed with exact error codes. Its OpenSearch Serverless-backed knowledge base has a filterable text field. A separate custom search benchmark lets the team inspect knn candidate settings and graph-memory behavior.

The team compares baseline SEMANTIC retrieval with HYBRID and several numberOfResults budgets. It also tests a deterministic query cleanup that preserves error codes. A decomposition experiment handles multi-part questions but measures its transformation and extra retrieval costs.

In the custom index test, explicit k and size prevent accidental per-shard result expansion, and KNNGraphMemoryUsage identifies memory pressure under load. The team tests normalized hybrid combinations against labeled evidence, rather than importing raw scores into one ranking.

It releases the configuration that improves retrieval p95 while retaining required evidence and permissions across both question classes. A faster low-candidate configuration that loses rare error-code passages is rejected.

Common exam traps

Equating Knowledge Bases numberOfResults with OpenSearch k.
Omitting size and assuming k bounds overall returned hits.
Measuring search while excluding query-rewrite latency.
Reading a smaller post-filtered result set as evidence of a faster index.
Adding shards to speed up one query without checking shard size and resource overhead.
Lowering candidate counts without checking required evidence recall.

What to remember

Optimize the complete retrieval path under a relevance gate.
Tune candidate budgets, index topology, and graph memory using their own mechanisms.
Preserve original query intent and exact identifiers.
Use supported hybrid paths and normalize distinct score scales.
Release only improvements that retain relevant authorized evidence.