AWSAWS Certified Generative AI Developer - Professional2026-103.1 Implement input and output safety controls
Enforce moderation before processing and publication
Completed by the knowledge check
Learning objective
Implement real-time input and output moderation with Amazon Comprehend, FM-based toxicity checks, Lambda, and Step Functions.
Core idea
Real-time moderation is a decision made before content crosses a protected boundary. An input gate decides whether a request can proceed to retrieval or generation. An output gate decides whether generated content can reach the caller. Running a detector after release provides observation, but it does not implement either gate.
Separate the detector from the decision. Amazon Comprehend returns toxicity signals. A foundation model (FM) can assess content under a task-specific rubric. Lambda can adapt requests, validate assessment results, and implement the release boundary. Step Functions can make the required ordering and failure paths explicit. The architecture remains responsible for thresholds, permitted outcomes, and deadlines.
Use distinct outcomes for allowed content, policy rejection, ambiguous assessment, and assessment failure. A low score and an unavailable detector are different evidence. Neither the wording of a classifier prompt nor a successful orchestration execution should silently turn missing evidence into approval.
How it works
Interpret Comprehend results within their scope
DetectToxicContent accepts LanguageCode and a list of TextSegments, each containing Text. The operation performs synchronous toxicity analysis and returns ResultList in correspondence with the input list. Each result contains overall Toxicity and category Labels, whose entries include Name and Score.
Use the category scores when your policy distinguishes kinds of harmful content. An application might reject a high-confidence threat while routing borderline offensive phrasing for review. Those thresholds and outcomes are application decisions; the service returns signals rather than a universal allow-or-deny policy. Keep a mapping from segment position to original content so the decision covers every submitted segment.
The request supports up to ten strings, each at most 1 KB, and at most 10 KB for the list. These are byte-size constraints, not character counts. Validate encoded sizes and split longer content without dropping text. Splitting also changes context: a threat can span a boundary, so evaluate the segmentation strategy on representative conversations instead of assuming independent snippets preserve meaning.
The operation’s prose documents English-only toxicity detection, even though the API’s LanguageCode enum lists other languages. Use en for this detector and route other languages to a separately validated path. A generic enum does not establish that this particular feature supports every listed language.
AWS’s availability change closes topic modeling, event detection, and prompt safety classification to new customers; AWS says other Comprehend features are unaffected. Do not confuse that notice with retirement of toxicity detection. Equally, do not assume toxicity detection supplies prompt-attack classification. Use the safety mechanism appropriate to that threat; Guardrails policy selection is covered in 3.1.1.
Constrain an FM assessment
An FM-based toxicity check can evaluate a domain-specific rubric, such as distinguishing a quoted abusive message from an instruction to threaten someone. Invoke the approved model through Converse, carrying the rubric in application-controlled instructions and the text as content to assess. Treat instructions inside the submitted text as data, and test attempts to redirect the assessor.
Require a small result contract: a verdict, relevant category, and evidence excerpt. On models that support Bedrock structured outputs, outputConfig.textFormat can constrain the JSON response. The schema constrains shape, not the assessment’s accuracy; see 3.1.4. A string named confidence is only the model’s generated claim unless you have calibrated its meaning against reviewed labels.
Validate the response before using it. Unknown verdicts, missing fields, incompatible model capability, and incomplete output take the assessment-failure route. Do not ask the same classifier repeatedly until it produces an allow verdict. If you allow a second assessment for ambiguous cases, define the maximum attempts and the terminal outcome first.
Use an FM assessor where the added context sensitivity earns its latency and operating cost. A separate toxicity model can itself be manipulated or misclassify benign content. Maintain reviewed test cases for quotations, reclaimed language, indirect threats, and adversarial instructions. Passing a second model’s judgment is evidence under that rubric, not an independent factual proof.
Place Lambda and workflow decisions at the boundary
Lambda adapters can validate language and size, call Comprehend or Bedrock, and return a normalized application result. Keep detector identity, policy version, scores, and verdict separate so the caller can distinguish a service result from the policy applied to it. The application-defined result is not an AWS response field.
A direct Lambda path fits a short input-check, generation, and output-check sequence. Step Functions fits a sequence whose required branches and recovery need to be visible as workflow state. The optimized Lambda integration uses arn:aws:states:::lambda:invoke and returns the function result under Payload. A raised function exception fails the workflow Task. Do not mistake a successful Task containing a business verdict of reject for a failed invocation.
After the assessment Task, a Choice state routes the validated verdict. Use a default route that withholds content when the result does not match an accepted contract. Without a matching choice or a Default, the workflow fails because it cannot transition to another state. Put output moderation after generation and before the publication step. Keep transient service retries bounded with Retry and route exhausted failures through Catch; workflow recovery is developed in 2.1.5.
The Choice state routes the validated verdict, its default withholds content, and only allowed content reaches generation, output moderation and the publication step.
For a synchronous request path, Synchronous Express workflows can start through StartSyncExecution, wait for completion, and return the result. Express workflows have a five-minute maximum duration and do not support callback or job-run integration patterns. The maximum is a platform limit, not a sensible interactive latency target. Standard workflows fit longer human-review paths; callback approval is covered in 2.1.9.
Key distinctions and trade-offs
Comprehend supplies defined toxicity outputs with documented language and payload boundaries. An FM assessor supplies flexible rubric reasoning that you must validate and calibrate. Combining them can improve coverage, but a disagreement needs an explicit rule rather than an arbitrary average of unrelated scores.
Lambda owns adapter behavior and release enforcement. Step Functions owns ordering and branching when you choose workflow orchestration. Adding both does not create a streaming moderation guarantee. Content already delivered cannot be recalled; withhold the answer until the required output gate passes.
Architecture / decision scenario
A customer-support system accepts English requests and generates draft replies. Its Lambda adapter first checks encoded text size, calls DetectToxicContent, and maps Toxicity and category scores to the reviewed policy. A high-risk request stops before generation.
A Synchronous Express workflow routes borderline content through an FM rubric assessment, validates its structured result, and uses Choice to allow, reject, or return a review-required outcome. Allowed requests proceed to generation. Both Comprehend and the domain assessor then evaluate the draft reply before a publication Task can run.
Non-English content follows a tested FM path instead of being sent to Comprehend under an unsupported language assumption. Detector errors and malformed assessor results withhold publication. Longer human review moves to a Standard workflow rather than stretching the synchronous request deadline.
Common exam traps
- Treating
Toxicityas an AWS-prescribed rejection threshold. - Inferring multilingual toxicity support from the generic language enum.
- Confusing the prompt-safety availability change with toxicity detection.
- Trusting a schema-valid FM verdict without calibration.
- Returning content before the output moderation Task completes.
- Treating a detector failure as a low-risk content result.
What to remember
Moderation requires a detector, a decision policy, and an enforced release boundary. Preserve Comprehend’s segment mapping and feature-specific limits. Validate FM assessments as untrusted outputs. Use Lambda for adapters and Step Functions for mandatory branches, with separate routes for rejection, ambiguity, and failure.
Knowledge check
A support system receives messages in English and Spanish and moderates them with DetectToxicContent, whose LanguageCode enum also lists es. Which language policy fits?
en , and route Spanish to a separately validated assessoren , since toxicity scoring is language-neutralAn FM toxicity assessor returns parseable JSON: {"verdict": "unclear", "confidence": 0.92}. The application’s contract accepts only allow, reject and review. What should the adapter do?
allow , because the confidence is above 0.9allow or rejectToxicity scoreallow turns the gate into a formality.In a Step Functions workflow, a Lambda Task returns {"verdict": "reject"} in its Payload, and the Task succeeds. A teammate expects the Task’s Catch to stop the content from being published. What does the workflow need?
Catch handles any Task whose result is a rejectionRetry on the Task, so the assessment repeats until it returns allowDefault that withholds contentCatch handles all of themCatch never sees it. A Choice state routes the verdict, and a Default that withholds content keeps an unexpected result from reaching publication or failing the workflow.A Lambda handler streams a generated reply to the caller while it sends the same reply to DetectToxicContent, and it rejects the reply if the score is high. What is wrong with this design?
DetectToxicContent assesses user input only, not generated output