Tune retrieval before reranking candidate evidence

Learning objective

Improve retrieval precision with keyword, vector, hybrid, semantic ranking, and reranking.

Core idea

For an architect tuning retrieval, separate finding candidates from ordering them. Keyword and vector retrieval produce candidate lists. Hybrid search combines lists. Semantic ranking reranks the candidates using their text. A final evidence-selection policy decides which authorized passages the agent receives. Improving one stage does not prove that the others work.

Azure AI Search exposes different mechanisms and scores for these stages. Keyword retrieval uses BM25, vector retrieval uses vector similarity, hybrid fusion uses Reciprocal Rank Fusion (RRF), and semantic reranking returns @search.rerankerScore. Do not interpret these scores as interchangeable probabilities of correctness.

See 2.2.8 and 2.2.9 for the corpus and representation. Query tuning cannot recover missing source text, incompatible embeddings, or evidence excluded before candidate retrieval.

How it works

Choose a candidate-generation method

Keyword retrieval is useful when the user’s terms identify the evidence directly: a product name, error code, policy identifier, or specific phrase. The request’s search string and eligible searchable fields establish the text query. Use searchFields to restrict scoring to the intended searchable fields, such as product titles and troubleshooting text, rather than unrelated boilerplate. BM25 ranking orders the matching text candidates; it does not independently determine whether a passage supports the requested conclusion.

Vector retrieval compares a query embedding with indexed embeddings. For a supplied vector, use vectorQueries.kind="vector", provide vector, and name the vector fields. Alternatively, an index configured with a vectorizer can use kind="text" and a query text value. The document and query representations must remain compatible, as explained in 2.2.9.

vectorQueries.k selects the nearest-neighbor count. exhaustive=true requests exhaustive nearest-neighbor search even on an HNSW-indexed field. A nearest neighbor can still be unrelated to the user’s actual need: the service can return the requested neighbors even when no strong evidence exists. Validate evidence relevance rather than treating a populated result list as an answer.

Fuse lexical and semantic-similarity candidates

A hybrid query includes both search and vectorQueries. Azure AI Search runs the text and vector components and combines their ranked lists with RRF. This handles requests where exact terminology and paraphrased intent both matter without requiring a direct comparison of BM25 and vector similarity scales.

RRF uses rank positions to create a combined @search.score. Its algorithmic constant is separate from the nearest-neighbor k in the vector query. Increasing the vector candidate count changes the candidate list; it does not configure the RRF constant.

The vector query’s weight changes its relative contribution during fusion. Increase or decrease it only after inspecting the cases where one candidate list overwhelms useful evidence from another. A heavily weighted semantic-similarity path can promote a broadly related passage over an exact identifier match. Vector weight applies only to vector queries; text has an implicit neutral weight of 1.0. For the text side, preview hybridSearch.maxTextRecallSize controls the BM25 candidate input to fusion (default 1,000 results; maximum 10,000), letting you adjust lexical influence without treating it as a vector weight.

Use top to control the number of returned hybrid results. The number retrieved for ranking and the number returned to the agent are different choices. Retrieving more candidates for fusion can improve coverage while returning fewer passages keeps the generation context focused.

Apply semantic reranking to the candidate text

Azure AI Search semantic ranker is a secondary ranking stage over an initial BM25- or RRF-ranked result. A query can request it with queryType="semantic" and semanticConfiguration naming the configured semantic fields. The ranker uses text even when candidates originated from vectors.

Only the top 50 candidates proceed to semantic ranking. Microsoft’s vector-query guidance therefore recommends k=50 when vector retrieval feeds semantic ranking. A small returned top can still follow this larger candidate pool. Do not set k=5 merely because the agent should receive five passages.

The two candidate lists meet in RRF, and only the top 50 fused candidates reach the semantic ranker before a smaller top is returned.

The semantic configuration identifies title, keyword, and content fields in priority order. The ranker’s summarization input accepts up to 2,000 tokens per document; the title and keywords each receive up to 128 tokens, with the remaining budget for content. Oversized or poorly prioritized text can hide the important qualification before reranking begins.

The returned @search.rerankerScore ranges from 0 to 4, with higher scores indicating greater semantic relevance. It is distinct from the initial @search.score. Semantic captions and optional answers extract text verbatim from the index; they are not generated explanations or proof that the application answered every clause.

Semantic ranker is available in select regions and incurs its own usage charges. Its core reranking does not imply that optional query rewrite is generally available: the semantic overview marks query rewrite preview. Likewise, the vector threshold parameter is preview and applies a minimum score before fusion. Do not present either as a universal prerequisite for semantic ranking.

Distinguish vector rescoring from text reranking

For compressed vectors, Azure AI Search also supports vectorSearch.compressions.rescoringOptions: it recalculates similarity using original, uncompressed vectors and reranks the initial vector candidates. defaultOversampling broadens that candidate pool before returning the requested neighbors. This repairs information loss from quantization; semantic ranker instead evaluates candidate text against the query.

Choose the mechanism for the observed failure. If compressed-vector retrieval loses a useful neighbor, evaluate rescoring and oversampling. If retrieved text is topically related but poorly ordered for the question, evaluate semantic ranking. Neither step makes an unauthorized or incomplete passage acceptable evidence.

Diagnose the failing stage before tuning

Inspect candidate membership first. If the correct passage never enters the first-stage pool, a reranker cannot promote it. Compare keyword, vector, and hybrid candidates on the same evaluation questions and inspect exact-identifier and paraphrase failures separately.

Then inspect ordering. If the useful evidence enters the pool but ranks below distractors, compare semantic reranking and field priorities. Check the returned source text for missing exceptions or version metadata before changing thresholds. A scoring change cannot repair an incomplete chunk.

Finally, inspect the evidence sent to generation. Deduplicate overlapping passages and apply the context budget in 2.2.3. Keep authorization enforced throughout; relevance ranking is not a permission check. If no passage supports the answer, use the application’s insufficient-evidence outcome rather than lowering every acceptance criterion.

Key distinctions and trade-offs

Vector similarity versus semantic ranking: Vector retrieval compares embeddings to find candidates. Semantic ranker reads candidate text to reorder an existing result set. Both use semantic information, but they occupy different stages.

Fusion versus reranking: RRF combines ranked lists. Semantic reranking applies language understanding to their resulting text candidates. Hybrid search does not automatically mean semantic reranking ran.

Candidate count versus response count: k helps determine vector candidates; top bounds returned hybrid results. A broader ranker input can coexist with a small agent context.

Keyword retrieval preserves exact terminology, vectors help with paraphrases, and hybrid search broadens coverage. Semantic ranking adds computation and latency while improving ordering for suitable text. Choose the combination from measured failures, not from the number of features enabled.

Architecture / decision scenario

A service assistant receives “Why does ZX-410 fail after a firmware update?” Its index contains exact error codes and descriptive troubleshooting passages. A keyword baseline finds code matches but misses paraphrased symptoms. A vector baseline finds related symptoms but sometimes prefers another product’s procedure.

The team uses a hybrid request containing search and vectorQueries, retains exact product-version constraints, and compares the fused candidates. It enables queryType="semantic" with the relevant title and content fields, uses k=50 for vector candidates, and returns a smaller top for evidence selection.

If the correct procedure is absent from the candidate pool, the team investigates ingestion and first-stage retrieval. If vector compression caused the miss, it evaluates rescoringOptions and defaultOversampling against the uncompressed baseline before changing text reranking. If it is present but poorly ordered, the team evaluates field priorities and reranker output. If the highest-ranked passage lacks the update-specific exception, it repairs chunking rather than calling the score a confidence guarantee. The agent receives a bounded, source-bearing evidence set or an explicit insufficient-evidence result.

Common exam traps

• Treating vector retrieval and semantic ranking as the same operation.
• Assuming hybrid fusion automatically invokes semantic ranker.
• Comparing BM25, RRF, and reranker scores on one numeric scale.
• Confusing the RRF constant with vectorQueries.k.
• Restricting semantic ranker’s candidate input to the final response size.
• Expecting reranking to retrieve a passage absent from its candidate pool.
• Treating an extractive semantic answer as generated verification.

What to remember

Tune candidate retrieval, fusion, reranking, and context selection separately. Use the identifiers and scores for the stage they belong to. Improve first-stage recall before asking a text reranker to improve precision, and preserve an insufficient-evidence outcome when the retrieved content cannot support an answer.

Knowledge check

References

• Microsoft, Create a vector query
• Microsoft, Relevance scoring using Reciprocal Rank Fusion
• Microsoft, Semantic ranking overview
• Microsoft, Create a vector index
• Microsoft, Create a hybrid query