Tune hybrid retrieval before accepting evidence

Learning objective

Combine semantic and keyword retrieval with hybrid search and relevant ranking thresholds.

Core idea

Hybrid retrieval gives you two ways to find evidence: meaning and literal wording. Semantic retrieval helps when a question paraphrases the source. Keyword retrieval helps when the decisive evidence contains a product identifier, error code, or unusual name. Combining them can improve the candidate set, but it does not establish that a passage answers the question.

For an architect choosing a retrieval policy, the decision has two parts. First, select a supported search strategy that finds the required evidence. Second, decide which returned passages are relevant enough to reach generation. A ranking score is an input to that second decision, not proof of correctness or permission.

How it works

Select the supported search path

For a custom Amazon Bedrock Knowledge Base, set retrievalConfiguration.vectorSearchConfiguration.overrideSearchType in Retrieve or the corresponding retrieval configuration in RetrieveAndGenerate. HYBRID searches both vector embeddings and raw text. SEMANTIC searches vector embeddings only. If you omit the override, Bedrock chooses the search strategy from the vector-store configuration.

Backend support matters. The AWS user guide documents hybrid search for Amazon RDS, Amazon OpenSearch Serverless, and MongoDB vector stores with a filterable text field. The API reference for KnowledgeBaseVectorSearchConfiguration describes the narrower OpenSearch Serverless case. These descriptions differ. Use OpenSearch Serverless with a filterable text field for a design supported by both descriptions; validate a proposed RDS or MongoDB configuration against the user guide and an actual retrieval test before adopting it. Do not assume every store from 1.5.5 accepts HYBRID.

Fully managed knowledge bases use a different configuration path. AWS documents hybrid retrieval as mandatory there; semantic-only retrieval is unavailable. They use managedSearchConfiguration, rather than the custom knowledge base’s vectorSearchConfiguration. A comparison between hybrid and semantic search therefore belongs to a supported custom configuration, not a fully managed knowledge base.

For a custom knowledge base, numberOfResults accepts 1–100 and sets the maximum number of source chunks to retrieve. It does not guarantee that many useful passages. Hierarchical chunking can return fewer results because retrieved children sharing a parent are replaced by that parent; see 1.5.1. Choose candidate depth from evaluation evidence rather than treating the largest accepted value as the safest setting.

Separate ranking from acceptance

Retrieve returns retrievalResults, including each result’s content, metadata, location, and score. AWS describes score as a relevance score useful for analyzing ranking. The vector-search configuration has no general minimum-relevance-score parameter. If your design requires a score cutoff before generation, apply that policy in application code after Retrieve; see 1.5.6 for the control boundary.

A cutoff should answer a specific operational question: does this passage provide enough evidence to be included for this query class? Label representative questions with the passages needed to answer them. Include exact identifiers, paraphrases, ambiguous names, and questions for which the corpus contains no answer. Evaluate semantic and hybrid retrieval with the same corpus, filters, and candidate depth. Amazon Bedrock Evaluations can run this comparison as two retrieve-only RAG evaluation jobs on one prompt dataset, each setting overrideSearchType in the job’s knowledgeBaseRetrievalConfiguration.vectorSearchConfiguration. Their built-in metrics, Builtin.ContextRelevance and Builtin.ContextCoverage, score the retrieved texts; context coverage needs ground truth in the dataset. The job measures retrieval quality, and your score cutoff remains an application policy that you calibrate against it.

Measure candidate recall before applying a threshold, then precision and answer coverage after applying it. Otherwise, an aggressive cutoff can appear successful because it removes nearly everything. Include correct abstentions and incorrect abstentions in the review. The application should distinguish an empty approved candidate set from a retrieval service error, even if both prevent generation.

Do not copy a numeric threshold from another index. A change in embeddings, retrieval strategy, or reranking changes the experiment you are evaluating. A score cutoff that works for a paraphrased policy question might suppress the only exact error-code match. Keep separate evaluation slices and adopt different policies only when their evidence and maintenance cost justify the distinction.

Preserve the hard constraints

Apply the trusted metadata filter during retrieval and retain the permission checks described in 1.4.6. Search type and relevance thresholds are quality controls. Neither authorizes a document. If a passage falls below your relevance boundary, remove it; if it fails authorization, exclude it regardless of how highly it ranks.

After thresholding, examine whether the remaining passages cover all parts of the question. A high-ranking troubleshooting introduction might mention the right product while omitting the actual repair instruction. If the identifier is ambiguous, ask for the product version. If the authorized corpus lacks the answer, abstain. These are application decisions, not undocumented Bedrock score meanings.

Key distinctions and trade-offs

The following decisions operate at different points:

Hybrid search can rescue exact terms that paraphrase-oriented retrieval misses. It can also add passages that share a term without answering the question. Increasing candidate depth improves opportunities to find evidence but adds material to inspect or rerank. Reranking is a separate relevance stage, covered in 1.5.8, and cannot recover evidence absent from the candidate set.

Retrieval relevance and answer grounding remain separate. A passage about a policy is relevant to the query; it might still fail to support a generated eligibility claim. Judge the retrieved set and the final answer independently.

Architecture / decision scenario

A support assistant must explain error E742 and answer paraphrased installation questions. Its custom knowledge base uses OpenSearch Serverless with a filterable text field. The team compares SEMANTIC and HYBRID with a fixed numberOfResults of 20 and the same product-version filter. Twenty is an experiment setting, not an AWS recommendation.

Hybrid retrieval finds the error-code repair passage more consistently, while some literal matches are unrelated release notes. The team labels both sets and calibrates an application cutoff over Retrieve results. It also checks whether an approved passage contains the requested repair evidence; a high score alone does not pass that review.

The application requests a version when the code has multiple meanings. It abstains when no authorized passage supports a repair. When a later index release changes score distributions, the team reruns the same slices before promoting the threshold. A move to a fully managed knowledge base requires a fresh baseline because search is always hybrid there and reranking defaults differ.

Common exam traps

• Assuming HYBRID works with every Knowledge Bases vector store.
• Treating an omitted search override as an explicit semantic-only choice.
• Expecting numberOfResults to guarantee that many returned passages.
• Inventing a Knowledge Bases minimum-score request parameter.
• Treating a relevance cutoff as a permission filter or grounding test.
• Transferring a threshold unchanged after changing the retrieval pipeline.

What to remember

Use hybrid retrieval when literal terms and semantic meaning both matter and the configuration supports it. Bound candidate depth separately from relevance acceptance. For an explicit pre-generation cutoff, inspect Retrieve results and implement the policy in the application. Calibrate on labeled evidence and preserve authorization as an independent requirement.

Knowledge check

References

• AWS, Configure and customize queries and response generation
• AWS, KnowledgeBaseVectorSearchConfiguration
• AWS, Query a knowledge base and retrieve data
• AWS, Use metrics to understand RAG system performance
• AWS, Creating a retrieve-only RAG evaluation job