AWSAWS Certified Generative AI Developer - Professional2026-103.1 Implement input and output safety controls
Verify answer claims against retrieved evidence
Completed by the knowledge check
Learning objective
Verify responses against Knowledge Base source passages using confidence signals and semantic similarity checks.
Core idea
Retrieval supplies candidate evidence; verification decides whether that evidence supports the answer. A high-ranked passage can discuss the right subject while contradicting a generated claim. A citation can identify a source without establishing that every statement in the answer follows from it. Verification must preserve the relationship between claims, source passages, and the actual question.
Use confidence signals according to what they measure. Knowledge Bases retrieval scores describe relevance to the query. Contextual grounding scores describe support in supplied evidence. Semantic similarity describes proximity in an embedding representation. These signals can help route a response, but none is a universal probability that the answer is true.
For consequential claims, require support for the claim that changes the decision. An answer can contain several well-supported background sentences and one invented eligibility rule. Averaging evidence over the answer can conceal that failure. Design the release policy around the unsupported consequential claim, not the most reassuring score.
How it works
Preserve the returned passages and source identities
The Bedrock Knowledge Bases Retrieve operation returns retrievalResults. Each result can contain content, location, metadata, and score. AWS defines the score as the result’s relevance to the query; it is not an answer-grounding confidence. Retain the actual passage alongside its source identity so later verification uses the evidence the generator received.
Keep authorization and source revision checks upstream. A highly relevant passage from the wrong tenant or an obsolete policy version is unsuitable evidence. Retrieval filtering and permission enforcement are covered in 1.4.6; provenance is covered in 1.4.5. Verification adds support checking after those boundaries, rather than trying to repair unauthorized retrieval with a confidence threshold.
RetrieveAndGenerate returns output.text and citations. A citation includes generatedResponsePart.textResponsePart, whose span identifies a generated text segment, and retrievedReferences containing source information. Use this mapping to inspect the passage associated with a particular answer segment. Do not replace it with a single source link attached to the entire answer.
A referenced passage may support only part of a sentence. Check qualifiers such as date, jurisdiction, product version, and exceptions against the claim. If one statement joins two facts, both need support. A valid source location establishes provenance; it does not establish that the model interpreted the passage correctly.
Use grounding confidence for source-relative support
Bedrock Guardrails contextual grounding can assess a response against a supplied source and query. With ApplyGuardrail, send the source in a text block qualified as grounding_source, the question as query, and the candidate answer as the content to guard, optionally qualified as guard_content. Use the same evidence and question that belong to the generated answer.
ApplyGuardrail returns assessments[].contextualGroundingPolicy.filters, whose entries include type, score, threshold, detected, and action. The GROUNDING signal evaluates support in the source; RELEVANCE evaluates whether the response addresses the query. Keep them separate when selecting answer, clarification, or abstention.
Policy configuration and threshold bounds are covered in 3.1.1. Here, the key choice is calibration. Assemble reviewed supported, contradicted, partially supported, irrelevant, and unanswerable cases. Run the complete retrieval-generation-verification path, and choose thresholds from the cost of unsupported answers and unnecessary abstentions. Re-evaluate after changing source content, chunking, prompts, models, or the verification policy.
The documented grounding check supports summarization, paraphrasing, and question answering and excludes conversational QA/chatbot use cases. Do not treat attaching it to a chat application as documented conversation-wide verification. If the architecture needs broader conversational support checking, it needs a separately evaluated application verification design.
Multiple grounding_source blocks are combined for evaluation, as are multiple query blocks. This means the aggregate check is not automatically a per-claim, per-passage verdict. Preserve claim-to-passage associations in the application. Where a supported assessment use case permits separate checks, assess a consequential claim against its relevant passages instead of assuming unrelated supporting text validates the whole answer.
Add semantic similarity as a supporting signal
A semantic check can compare a claim with candidate source passages using embeddings. Amazon Titan Text Embeddings V2, identified as amazon.titan-embed-text-v2:0, is documented for semantic similarity as well as retrieval. It produces vectors; the actual similarity computation happens in your vector database or application, not inside the embedding model.
Use a consistent model, vector representation, and distance calculation for the two texts. The embedding contract and index metric choices are covered in 1.5.3 and 1.4.7. For response verification, the additional responsibility is to compare the generated claim with the source rather than interpret query-to-document similarity as answer support.
Similarity can find likely support passages or flag an answer that diverges from the available material. It does not establish entailment. The sentences refunds are permitted and refunds are not permitted share much of their subject and wording while expressing opposite rules. Numbers, negation, and exceptions therefore need explicit evidence review even when a similarity score is strong.
Choose any similarity threshold on reviewed claim-passage pairs from the deployed corpus. Do not copy a retrieval threshold into a claim-verification gate or combine cosine similarity and grounding confidence as if they had the same units. Keep the signals and their failure reasons separate so evaluation can show which check improved the decision.
Key distinctions and trade-offs
Retrieval relevance answers which passages are promising. Citation mapping answers which source is associated with a generated segment. Grounding confidence estimates support relative to supplied evidence. Semantic similarity finds related meaning. Their different roles are useful precisely because they are not interchangeable.
Source-grounded accuracy is also different from external truth. A response can faithfully reproduce an outdated or incorrect document. Verification therefore needs both a valid evidence set and faithful use of that set. More passages can improve coverage, but conflicting versions require an explicit authority rule rather than a larger context window.
Claim-level checks add processing and orchestration work. Apply them according to consequence: a wrong office-location description and a wrong coverage-exclusion rule have different error costs. Establish the supported unit of assessment and the release rule before optimizing verification latency.
Architecture / decision scenario
A single-question warranty assistant retrieves approved product-policy passages. It retains content, source location, revision metadata, and retrieval score, then generates a cited answer. The answer says a product is covered for three years, but the cited current passage says two years with a specific exclusion.
The system uses citation spans to locate the consequential claim and its referenced passage. Semantic similarity is high because both texts discuss the same warranty. That signal helps find the comparison; it does not approve the claim. A contextual grounding assessment receives the actual question, approved source text, and candidate answer, and the application checks the supported duration and exception explicitly.
An unsupported duration withholds the answer. Missing product-version information can trigger one clarification when it changes retrieval; absent authoritative evidence produces abstention. A strong retrieval score never overrides the unsupported claim. Regression cases include negation, conflicting policy revisions, uncited sentences, and accurate but irrelevant answers.
Common exam traps
- Calling
retrievalResults[].scorea probability of answer correctness. - Assuming a citation supports every nearby claim.
- Treating semantic similarity as entailment, especially for negation or numbers.
- Combining unrelated confidence scales into an unexplained average.
- Assuming combined grounding-source blocks provide per-passage verdicts.
- Treating source-grounded text as proof that the source itself is authoritative or accurate.
What to remember
Verify claims against the exact authorized passages used for generation. Preserve citation mappings and distinguish retrieval relevance, grounding support, and semantic similarity. Calibrate release decisions on reviewed outcomes, and withhold consequential claims whose evidence does not support them.
Knowledge check
A warranty answer states three years of coverage. The passage it cites has a retrieval score of 0.91 and states two years. What should the release policy do?
A verifier embeds each claim and its cited passage with Amazon Titan Text Embeddings V2. The claim “Refunds are permitted after 30 days” and the passage “Refunds are not permitted after 30 days” have a cosine similarity of 0.94. What should the verifier conclude?
A verifier sends five passages as separate grounding_source blocks, with the question and the full answer, to ApplyGuardrail. The GROUNDING check passes, yet one eligibility claim is not supported by the passage it cites. How can the check pass?
grounding_source blocks are evaluated together, not passage by passagegrounding_source blockguard_content qualifier, so it was not assessedGROUNDING measures whether the answer addresses the questiongrounding_source values, so support anywhere in the set can carry a claim whose own passage does not support it. Per-claim verification needs the application to keep claim-to-passage associations and check consequential claims against their own evidence.A response passes the grounding check against its supplied source, but that source is last year’s policy, which has since been replaced. What has to change for current-policy answers?