Tune grounding thresholds for safe coverage

Learning objective

Tune grounding thresholds to balance answer coverage, clarification, abstention, and hallucination risk. This lesson focuses on the decision policy around grounding scores, not retrieval-parameter tuning.

Core idea

A grounding threshold converts a continuous support signal into product behavior. Do not treat it as a universal truth boundary. Calibrate it on representative data and combine it with an explicit policy:

strong support -> answer
partial or ambiguous support -> clarify or qualify
insufficient support -> abstain or escalate

Raising a threshold usually reduces unsupported answers but also suppresses some useful answers. Lowering it increases coverage and the risk of accepting weakly supported claims.

How it works

Google Cloud’s check grounding API returns an overall support score from 0 to 1, plus claim-level citations. Its citation threshold also ranges from 0 to 1; a higher value yields fewer but stronger citations. Agent Search can return support scores and restrict answers to those that meet a configured support-score threshold, as described in Get answers and follow-ups.

Build a calibration set with real questions, approved evidence, and reviewed labels for supported, partially supported, conflicting, and unsupported answers. Include answerable and unanswerable requests, access-filtered cases, and high-risk topics. Run the complete retrieval and generation pipeline because a threshold cannot repair missing or poor evidence upstream.

For candidate thresholds, measure at least:

• the share of acceptable answers returned;
• the share of returned answers containing unsupported claims;
• correct abstention on unanswerable requests;
• useful clarification when the request is underspecified;
• behavior by topic, language, user group, and risk class.

Plot or inspect the confusion matrix at each threshold. Choose from business costs, not a round number. A low-risk product-help assistant may tolerate a qualified answer with partial support. A legal or safety workflow may require a higher threshold and human escalation.

Do not force every score into answer or refuse. A middle band can request a missing product version, jurisdiction, or date. Clarification is useful only when the missing input can change retrieval. If evidence is absent, repeated clarification creates a loop; abstain and provide a safe next step.

Apply decisions at the right granularity. An answer can have a strong aggregate score while one consequential claim is unsupported. Inspect claim-level support and citations when individual claims carry different risks. Monitor score distributions and reviewed outcomes after changes to models, prompts, sources, or retrieval.

Key distinctions and trade-offs

Retrieval relevance versus grounding support: Relevance asks whether evidence relates to the query. Grounding asks whether evidence supports generated claims. A relevant passage may not entail the answer.

Citation threshold versus answer threshold: A citation threshold controls which support links appear. An answer policy decides whether to return, qualify, clarify, or abstain. Hiding weak citations does not make the answer safer.

Abstention versus failure: A correct abstention is a successful safe outcome when evidence is insufficient. A system failure means required processing did not complete reliably.

A single global threshold is simple but can hide different error costs across domains. Per-risk policies improve safety but require more labeled data, governance, and monitoring.

Architecture / decision scenario

An employee benefits assistant supports general navigation and regulated coverage questions. Evaluation shows that a medium score often reflects an omitted country or plan year.

The team defines three bands. Strong claim-level support returns a cited answer. The middle band asks for country and plan year once, then reruns retrieval. Low support or conflicting policy versions produces an abstention and HR contact. Coverage decisions use a stricter threshold than office-location questions.

Only the middle band returns to retrieval, and it does so once.

After each index or model release, the team reruns the labeled set and reviews production samples near both boundaries.

Common exam traps

• Treating 0.5 or another default value as universally correct.
• Confusing retrieval similarity, citation confidence, and grounding support.
• Optimizing answer rate without measuring unsupported returned answers.
• Using one aggregate score while ignoring an unsupported high-impact claim.
• Repeatedly asking for clarification when no user detail can produce evidence.
• Assuming threshold tuning can fix stale, unauthorized, or missing source data.

What to remember

• A threshold implements a risk decision; it does not establish truth.
• Calibrate on labeled, representative, and adversarial pipeline outputs.
• Use answer, clarification, qualification, and abstention as distinct outcomes.
• Inspect claim-level support for consequential answers.
• Recalibrate after model, prompt, retrieval, or corpus changes.

Knowledge check

References

• Google Cloud, Check grounding with RAG
• Google Cloud, Get answers and follow-ups
• Google Cloud, Return grounding support scores
• Google Cloud, Show only well-grounded answers