Turn similar passages into usable evidence

Learning objective

Explain embeddings, vector similarity, retrieval, and evidence grounding.

Core idea

For an architect building a grounded assistant, a relevant-looking search result is a candidate source, not a verified answer. Embeddings represent content numerically, vector similarity ranks nearby representations, retrieval returns candidate material, and grounding uses that material to support a response. Each stage answers a different question.

A passage can resemble the question while describing the wrong product version or omitting the requested exception. Good grounding therefore requires both retrieval quality and a claim-to-evidence check. Model selection belongs in 0.1; this lesson explains how an embedding model participates in the evidence path rather than generating the final answer.

How it works

Represent content in an embedding space

An embedding is a vector of numbers produced from content by an embedding model. Its practical role is to make similarity comparisons possible. Azure OpenAI’s Python example calls client.embeddings.create() with input and model, and reads the vector from response.data[0].embedding. That vector is not a prose response, a source citation, or a probability that the input is true.

Index content and encode queries with the same embedding model. Microsoft recommends this explicitly for Azure AI Search vector queries. Equal vector lengths are necessary for compatible field storage, but equal lengths alone do not establish a shared semantic space. A query encoded with an unrelated model can produce misleading rankings even if its array fits the field.

Preserve the original content alongside the vector. Azure AI Search’s Foundry tool requires retrievable text containing the content to cite and a retrievable source-URL field. A vector without its corresponding passage can help find a record but cannot show the reader what supports the answer. Chunk identifiers and source metadata should survive ingestion so you can investigate which material reached generation.

Rank candidates by similarity

Vector search compares a query vector with stored vectors using a configured metric. Azure AI Search supports cosine, euclidean, and dotProduct in the index’s vectorSearch configuration; Microsoft recommends cosine for Azure OpenAI embeddings. Cosine similarity compares vector direction, Euclidean distance compares separation, and dot product compares the product-sum of components.

In a search request, vectorQueries.kind: "vector" supplies a vector array through vectorQueries.vector. vectorQueries.fields selects the target vector fields, and vectorQueries.k asks for a number of nearest matches. With a configured vectorizer, kind: "text" lets the service vectorize a text query. Those modes differ in who generates the query embedding, not in whether the search needs a compatible embedding space.

The vectorQueries.exhaustive option invokes exhaustive nearest-neighbor search, including for a field indexed using hierarchical navigable small world (HNSW). Exhaustive search provides a useful comparison when investigating approximate retrieval quality, but performs more comparison work. Do not confuse an algorithm’s ability to find nearby vectors with a guarantee that the nearby documents answer the question.

Nearest-neighbor search can return off-topic passages when they are the closest available candidates. Vector search returns the requested k nearest neighbors even when they are not very similar, up to the number of documents available in the index. The k parameter is a result count, not a relevance threshold. Filtering can further change what is eligible. A nonempty result list is therefore insufficient evidence of answerability.

Retrieve text and combine signals

Retrieval returns the selected source material and metadata to the application or agent. Full-text search is useful for exact terms; vector search adds semantic similarity. A hybrid query combines text and vector result sets. Azure AI Search uses reciprocal rank fusion (RRF) when combining multiple ranked sets, including hybrid results.

The Foundry AzureAISearchTool makes those choices concrete through index_name, query_type, top_k, and filter. Supported query types include simple, vector, semantic, vector_simple_hybrid, and vector_semantic_hybrid. The documented defaults are vector_semantic_hybrid and top_k of 5. These tool settings are distinct from the search REST request’s vectorQueries.k; do not assume a default observed in one interface defines the other.

Use filters to constrain the intended source set, such as an allowed product version or corpus partition. The tool’s filter applies to all queries the agent makes against that index. Establish trusted filter values in the application rather than letting retrieved text expand the corpus. User entitlement and document-access enforcement require their own design; see 0.6 and 1.2.4.

Ground claims in the retrieved evidence

Grounding connects generated claims to source content. Foundry’s search tool returns inline citations, and its response examples expose annotations with type: "url_citation". Preserve those annotations in the consuming interface so readers can inspect the supporting source. A formatted link alone does not establish that the cited passage entails every adjacent claim.

Review whether the evidence actually covers the requested entity, condition, and version. If the retrieved passage explains one plan’s refund rule, it cannot establish another plan’s eligibility. If evidence conflicts or leaves a required condition unanswered, the application should choose an explicit incomplete-evidence outcome rather than treating fluent generation as resolution. Similarity and fusion only select candidates; the evidence check decides between a grounded answer and an incomplete-evidence outcome.

That outcome is an application contract, not a universal Azure search setting. Retrieval parameters control candidate selection; they do not prove factual entailment. Similarity scores also depend on the ranking path. An RRF score from hybrid retrieval is not interchangeable with a single-vector similarity score or a calibrated confidence percentage. Azure AI Search’s vector-query threshold parameter is documented as preview. Microsoft recommends minimum score thresholds only for a pure single-vector query; hybrid queries have smaller, more volatile RRF score ranges. A threshold does not establish factual support.

Key distinctions and trade-offs

Separate the stages when evaluating a design:

Increasing candidate count can improve coverage but also introduces irrelevant context. Hybrid retrieval can help with both exact terminology and semantic paraphrases, while making scores dependent on fusion. Keeping source text and metadata costs storage but makes answers auditable. Evaluate retrieval and answer support separately: Foundry’s Retrieval evaluator measures retrieval effectiveness, while Groundedness scores response support in retrieved context from 1–5 using a model-based judgment; see 3.1.6. A good ranking result must not hide unsupported generation.

Architecture / decision scenario

A service assistant answers warranty questions for two product generations. A customer asks whether damage from a particular installation method is covered. Both generations’ documents contain nearly identical descriptions, but their exclusions differ.

The ingestion path stores compatible embeddings, retrievable passages, and source URLs. The application selects the trusted product generation and configures the search tool’s filter accordingly. It uses vector_semantic_hybrid to combine descriptive similarity with exact installation terminology, and chooses top_k from measured evidence coverage rather than assuming five passages always suffice.

The response must identify the applicable exclusion and retain the tool’s url_citation annotations. If the retrieved passages discuss installation but never state coverage, the application returns an incomplete-evidence outcome. Raising the tool’s top_k, raising k in a direct search request, or switching to exhaustive vector search can diagnose retrieval gaps, but none establishes the missing policy. The team tests the candidate passages and final supported claims independently.

Common exam traps

• Treating embeddings as generated answers or truth probabilities.
• Mixing document and query embeddings from unrelated models because their dimensions match.
• Reading k as a minimum relevance requirement.
• Comparing hybrid RRF scores directly with single-vector similarity scores.
• Assuming a citation proves every claim in the sentence it accompanies.
• Treating a preview vector threshold parameter as a universal grounding control.

What to remember

Embeddings make content comparable; retrieval supplies candidate evidence. Preserve compatible vectors, retrievable passages, and source metadata. Grounding requires actual support for the generated claim, even when search returns nearby results and the answer includes citations.

Knowledge check

References

• Microsoft, Generate embeddings with Azure OpenAI
• Microsoft, Vector search in Azure AI Search
• Microsoft, Create a vector query in Azure AI Search
• Microsoft, Connect to Azure AI Search
• Microsoft, Built-in evaluators