Skip to lesson

Google CloudProfessional Agentic Architect2026-093.2 Integrating enterprise domain knowledge

Apply reranking after high-recall retrieval

Completed by the knowledge check

Learning objective

Apply reranking when the first-stage retriever finds the required evidence but does not place it near the top. This lesson focuses on the second-stage ranking decision, not on embedding selection, retrieval-parameter tuning, or retrieval product selection.

Core idea

Use a retrieve wide, rerank narrow pipeline:

  1. A fast retriever selects a candidate set of size K from the full corpus.
  2. A more precise reranker assigns new query-specific scores to those candidates.
  3. The pipeline sends only the best N candidates to the generator, where N is smaller than K.

The candidate set creates a recall ceiling. A reranker can reorder or remove retrieved candidates, but it cannot recover a relevant chunk that the retriever did not return. Apply reranking only after evaluation shows that relevant chunks usually appear within the K candidates while irrelevant or weakly relevant chunks occupy the first N positions.

This sequential design avoids running an expensive ranking model across the entire corpus. Google Cloud describes retrieval as finding a subset and ranking as ordering that subset because ranking every available document can be computationally expensive in its retrieval and ranking overview.

How reranking works

Diagnose an ordering problem

Build a labeled set of representative queries and relevant chunks. Measure at least two cutoffs:

  • Recall@K measures how much relevant evidence appears anywhere in the reranker input.
  • Precision@N measures how much of the final, smaller result set is relevant.
  • Normalized discounted cumulative gain (NDCG)@N measures ordering quality and gives more weight to relevant results near the top.

The useful reranking pattern is high-enough Recall@K with weak Precision@N or NDCG@N. Google Cloud’s search-quality evaluation similarly reports recall, precision, and NDCG at several cutoffs and recommends representative query sets; see Evaluate search quality.

If Recall@K is low, repair the first stage instead. Review the embeddings, query representation, metadata, filters, thresholds, and K. Making the ordering inside an incomplete candidate set more accurate does not fix missing evidence.

Score candidates with richer query–chunk interaction

Embedding retrieval commonly encodes the query and chunks separately, then compares their vectors. This supports efficient search over a large corpus. A common reranker is a cross-encoder, which processes each query–chunk pair together and produces one relevance score. The joint processing can model fine-grained interactions, but it requires work for every pair and is therefore slower. The Sentence Transformers Cross Encoder guidance documents this quality–latency reason for reranking only top-k results.

A cross-encoder is not the only reranker. A managed semantic ranking service or an LLM can also assess candidates. For example, RAG Engine supports both a dedicated ranking service and an LLM reranker. The dedicated service targets low-latency relevance scoring, while an LLM reranker adds a separate model call; see Reranking for RAG Engine.

Pass the same user intent used for retrieval, stable candidate IDs, and enough title and chunk content to preserve meaning. Respect the ranker’s input limit: the Google Cloud ranking API truncates record content beyond the selected model’s limit. It accepts an existing candidate list, scores every supplied record, and can return only the requested top N, as described in Improve search and RAG quality with ranking API.

Validate the complete pipeline

Compare the baseline and reranked systems on the same held-out queries. Track Recall@K before reranking, Precision@N and NDCG@N after reranking, end-to-end answer quality, empty-result rate, latency, and cost. Segment results by query type, language, and corpus area so that an average does not hide regressions.

Tune K and N together. A larger K gives the reranker more opportunities to find the best evidence, but increases scoring work. A smaller N reduces prompt noise and tokens, but can remove complementary passages. Do not ship a reranker merely because its offline relevance score looks better; confirm that the generator receives better evidence and produces better grounded answers.

Key distinctions and trade-offs

Retrieval versus reranking: Retrieval searches the corpus and determines which candidates exist. Reranking rescores them and determines their order within that fixed set.

Reranking versus filtering: Reranking expresses relative relevance. A metadata or authorization filter expresses eligibility. Apply access controls before the reranker; a high relevance score never makes a prohibited chunk permissible.

Reranking versus generation: A reranker selects evidence. It does not synthesize an answer or prove that a later answer is grounded.

Ranking score versus probability: Use scores to order candidates under the same ranker configuration. Do not assume that an arbitrary score is a calibrated probability or compare score values across different models without validation.

The main trade-off is ranking quality versus serving cost. Candidate count, candidate length, model size, and remote calls all affect latency and cost. Batching, caching stable query–candidate results where safe, or choosing a faster ranker can reduce cost, but each optimization requires quality testing.

Architecture / decision scenario

A maintenance assistant searches manuals for several generations of industrial pumps. For the query “reset the seal pressure alarm after replacing the sensor,” vector retrieval returns 40 permitted chunks. Evaluation shows that the correct procedure appears in the candidate set for 96% of similar queries, but generic alarm descriptions often fill the five positions sent to the model.

Because candidate recall is already sufficient, the team does not enlarge K or replace the embedding model. It sends the 40 candidates to a semantic reranker, then supplies the top five to generation. Product-family and access filters run before retrieval and remain mandatory. The team validates Recall@40, Precision@5, NDCG@5, answer correctness, p95 latency, and cost on held-out queries.

Precision@5 and answer correctness improve within the latency budget. The team keeps the reranker. If the correct procedure had been absent from the 40 candidates, this design would have hidden the actual retrieval problem rather than solved it.

Common exam traps

  • Adding reranking when relevant evidence is missing from the candidate set.
  • Assuming reranking increases the candidate set’s recall. It can only change the order or selection of existing candidates.
  • Sending the entire corpus to a precise ranker instead of using fast retrieval first.
  • Treating a larger retrieval K as free; it increases reranking work and may increase latency and cost.
  • Replacing authorization or metadata filters with relevance scoring.
  • Measuring only Recall@K and overlooking whether useful evidence reaches the small context cutoff N.
  • Evaluating reranking scores without checking final grounded-answer quality.

What to remember

  • Use reranking when relevant evidence is present in the candidate set but ordered poorly.
  • Treat Recall@K as the reranker’s ceiling; fix retrieval when that ceiling is too low.
  • Retrieve wide with an efficient method, rerank the candidates with a more precise scorer, and send a smaller top N to generation.
  • Keep eligibility and authorization controls separate from relevance ranking.
  • Validate ranking quality, grounded-answer quality, latency, and cost together.

Knowledge check

References