AWSAWS Certified Generative AI Developer - Professional2026-100 Prerequisites
Separate sampling controls from completion limits
Completed by the knowledge check
Learning objective
Configure temperature, top-p, top-k, token limits, and stop sequences.
Core idea
For an engineer configuring generation, two questions matter: which candidate tokens the model considers, and when it stops producing output. Temperature, top-p, and top-k influence sampling. Token limits and stop sequences constrain completion. Neither category supplies missing evidence or validates the answer.
Configure the controls supported by the exact model and API. Amazon Bedrock Converse provides a shared inferenceConfig with temperature, topP, maxTokens, and stopSequences. Model-specific controls belong in additionalModelRequestFields when supported. A common interface does not make all controls valid for every model. This lesson explains their semantics; see 4.2.6 for controlled parameter experiments.
How it works
Temperature changes the sampling distribution
During text generation, the model assigns probabilities to possible next tokens. Temperature changes the shape of that distribution. A lower value favors higher-probability candidates; a higher value gives lower-probability candidates more opportunity. This affects variation, not the factual evidence available to the model.
For Converse, inferenceConfig.temperature has an API range from 0 to 1, and its default depends on the selected model. Start with a documented supported setting for the task. A constrained extraction task and a brainstorming task can have different useful settings, but the application must still validate each result. Do not describe temperature as a confidence score or probability that a response is correct.
Top-p and top-k choose different candidate pools
Top-p, also called nucleus sampling, retains a pool based on cumulative probability mass. With topP: 0.8, the candidate pool covers the most likely part of the distribution up to the chosen mass. The number of tokens in that pool can change at every generation step. Converse accepts inferenceConfig.topP in the range 0 to 1; the model supplies the default.
Top-k retains a fixed number of the most likely candidates. If k is 40, it limits the pool to the 40 highest-ranked tokens at that step. Forty is a teaching example, not a recommended production value. Top-k does not mean “the top 40 percent,” and top-p does not specify a fixed count.
Suppose one next-token distribution concentrates most probability in three candidates, while another spreads it across dozens. A fixed k selects the same number in both distributions. A fixed p can select different numbers because it follows the probability mass. The settings express different restrictions, so matching their numeric values is meaningless.
Converse has no topK member in its shared InferenceConfiguration. For models that support an additional top-k field, use the documented model-specific name through additionalModelRequestFields. AWS documents native Anthropic Claude Messages fields as top_k, top_p, and temperature. Native names and common Converse names are not interchangeable request keys.
Also verify parameter combinations. AWS documents Claude Sonnet 4.5 and Claude Haiku 4.5 as permitting either temperature or top_p, but not both. That is a specific model restriction, not a universal Bedrock rule. A configuration builder that always emits both fields can produce an invalid request even when each individual value lies within a valid range.
Token limits bound generated length
Converse’s inferenceConfig.maxTokens is the maximum generated response length in tokens. Its API minimum is 1, while the selected model determines the supported maximum. If omitted, the API documentation says its default is the model’s maximum allowed value. Set a deliberate cap when the product has a bounded answer requirement.
The cap is not a target. A model can finish before reaching it. Increasing it does not instruct the model to add detail, and lowering it does not guarantee a concise but complete answer. It can instead cut a result before its conclusion or closing structure. A “brief answer” instruction and an enforced output cap serve related but different purposes.
For a native Anthropic Messages request, the required field is max_tokens, not maxTokens. In a Converse response, inspect stopReason; max_tokens indicates that generation stopped at the output limit. Do not accept an incomplete record merely because it arrived without a transport error. See 0.2 for the separate context-window boundary.
Stop sequences bound textual continuation
Converse’s stopSequences is a list of character sequences that stop generation when encountered. Native Anthropic Messages uses stop_sequences. This control is useful when a documented output convention has a delimiter, but it is not a schema validator or a guarantee that the content before the delimiter is correct.
Choose delimiters that cannot normally occur in valid content. A common word, newline, or punctuation mark can stop an otherwise useful answer too early. Check the selected model’s treatment of matched delimiters rather than assuming whether they appear in returned text.
Converse reports stopReason: stop_sequence when a stop sequence ends generation. For native Anthropic responses, AWS documents stop_reason: stop_sequence and stop_sequence containing the matched sequence. This evidence distinguishes an intentional delimiter stop from output-token exhaustion. Neither event by itself proves application-level completion.
Key distinctions and trade-offs
Sampling versus grounding: Sampling controls which plausible continuation is chosen. It cannot make unsupported information into source-backed information.
Common API versus native body: Converse uses topP, maxTokens, and stopSequences. Native Claude Messages uses top_p, max_tokens, and stop_sequences. Keep names attached to their interfaces.
Variation versus repeatability: Lower temperature favors more likely tokens. Treat repeatability as something to measure, rather than a factual correctness contract. See 0.5 for nondeterminism and hallucinations.
Length versus structure: A token cap and a textual delimiter can interrupt an output. If a consumer requires structured records, use an eligible structured output mechanism and retain completion and business validation; see 3.1.4.
Architecture / decision scenario
A team builds two functions using a supported Claude Messages model: extract a short incident summary and propose alternative subject lines. Its shared request builder copies the same arbitrary settings into every model invocation.
For extraction, the team selects one supported sampling control, keeps a bounded output allowance, and validates whether the incident summary retains the required facts. For subject lines, it evaluates a more varied setting against the required format and suitability. It compares top-p and top-k as different candidate-pool controls, rather than treating them as equivalent sliders.
When moving the request builder to Converse, the team maps the common fields to inferenceConfig and places supported top_k in additionalModelRequestFields. If the chosen model is Claude Sonnet 4.5, the builder does not emit both temperature and top-p. It chooses a rare delimiter only for the function that needs textual stopping and checks stopReason for both stop_sequence and max_tokens. The application validates content after either stop, and does not mistake a lower-temperature invented fact for an accurate extraction.
Common exam traps
- Reading temperature as answer confidence.
- Treating top-k as a percentage or top-p as a fixed candidate count.
- Putting
topKin Converse’s sharedinferenceConfig. - Reusing native parameter names in the common Converse configuration.
- Assuming every model accepts all sampling controls together.
- Treating
maxTokensas a required output length or context-window setting. - Using a stop sequence as proof of valid structured output.
What to remember
Sampling selects from candidate continuations; length and stop controls determine when generation ends. Keep parameter names, ranges, and allowed combinations bound to the selected model and API. Inspect the stop reason and validate the result independently of the settings that produced it.
Knowledge check
An extraction response invents a field absent from the source. A teammate proposes lowering temperature as the correctness gate. Which interpretation should guide the design?
At some generation steps a model’s probability is concentrated in three candidates; at others it is spread across dozens. The team wants the sampler to consider the same number of candidates at every step. Which control expresses that restriction?
You are moving a supported model’s top_k setting into a Converse request. Where should it go?
inferenceConfig as topKinferenceConfig as top_kadditionalModelRequestFields as top_kadditionalModelRequestFields as topKInferenceConfiguration has no top-k member, so a model-specific control goes in additionalModelRequestFields under the name the model documents — top_k for Anthropic Claude. Common Converse names and native names are not interchangeable keys.A request builder for Claude Sonnet 4.5 on Amazon Bedrock sets both temperature and top_p, each within its documented range, and its requests are rejected. What should the team change?
inferenceConfigtop_k so the sampling controls stay consistenttemperature or top_p , but not both. Valid individual values do not make an invalid combination valid; this is a restriction of those models, not a universal Bedrock rule.Valid answers may contain the word END, and the request uses END as a stop sequence. Some responses stop before their required conclusion. Which change addresses this stopping defect?
maxTokens and keep END as the stop sequenceEND as the stop sequenceEND with a sequence valid content never containsEND as the stop sequence