AWSAWS Certified Generative AI Developer - Professional2026-104.1 Implement cost optimization and resource efficiency strategies
Reduce tokens without discarding required meaning
Completed by the knowledge check
Learning objective
Reduce token consumption using context pruning, prompt compression, and maximum-token response limits.
Core idea
Token reduction has three different targets: unnecessary information admitted into context, unnecessarily verbose representations of useful information, and unnecessary generated output. Pruning selects information. Compression changes its representation. A maximum-token setting bounds generation. Apply each control where it acts, then verify that the shorter request still supports the task.
For an architect reducing Bedrock inference cost, the goal is lower total usage per accepted response. Removing evidence that causes an answer to fail can trigger another retrieval and generation attempt. That apparent saving on the first invocation can increase both latency and total tokens. Measure the whole request, using the accounting boundary in 4.1.1.
Bedrock provides model-specific token counting and explicit generation limits. It does not make your application’s definition of dispensable context or equivalent meaning automatic. Preserve the distinction between AWS controls and the pruning and compression policy you implement before the invocation.
How it works
Prune by contribution to the task
Identify the payload components that consume input: task instructions, user content, conversation history, retrieved evidence, and tool results. Mark which facts and dependencies must survive. A shorter prompt that loses a relevant qualification, authorization boundary, or source identifier is not an equivalent prompt.
Prune duplicated evidence, irrelevant passages, obsolete intermediate results, and repeated formatting instructions before cutting required context. The application determines relevance and dependency; Bedrock Converse.messages and system carry the resulting request. Do not confuse a smaller message list with a supported semantic guarantee. Conversation ordering and tool-message dependencies remain important; see 1.3.9. Permission-filtered context assembly remains the boundary described in 1.5.11.
Establish a token budget for each component rather than one undifferentiated truncation rule. A source-grounded answer can reserve space for authoritative evidence and reduce conversational repetition first. A task that depends on an earlier user constraint must retain that constraint even when the surrounding dialogue is removed. Where several passages state the same fact, preserve the passage that carries the needed source and qualification.
Recount the assembled request for the actual target model. The Bedrock CountTokens API estimates input token usage before inference and incurs no charge. Tokenization is model-specific, so a character count or a token count from another model cannot establish that the selected payload meets this model’s budget. AWS also documents models that do not support CountTokens on bedrock-runtime, with a bedrock-mantle token-counting path for specified Anthropic models. Check the selected model’s support rather than assuming one endpoint works everywhere.
Compress representation while preserving evidence
Compression retains the information you need in a shorter representation. Remove redundant prose, collapse repeated instructions, and use concise task-specific descriptions. If you replace historical messages with a summary, define the facts the summary must preserve: decisions, unresolved questions, user constraints, and evidence references. Compression of a retrieved passage must not erase the distinction between a claim and its supporting source.
A model-generated summary is another inference step. Include its input and output tokens in the request ledger and evaluate whether later reuse repays that work. For a single short request, paying for compression can cost more than sending the original. For repeated long conversations, a reusable summary might reduce subsequent input substantially, but correctness still needs evaluation.
Compression shortens the representation; Bedrock prompt caching can reuse an eligible repeated prefix, with cache hits reducing input cost without shortening it. See 4.1.6 for caching.
Treat compression as a tested transformation, not a universal percentage target. Compare original and compressed requests on representative cases with expected facts, citations, and response contracts. Include cases where a short qualifier changes the answer, such as an exclusion or effective date. A high average quality score does not justify silently losing rare but consequential constraints.
Track the transformation version with each invocation record. If an input change lowers usage.inputTokens but raises rejection or follow-up rates, you need to identify which compression policy produced it. This version is application evidence; it is not a Bedrock token-counting feature.
Bound generation explicitly
For Converse, inferenceConfig.maxTokens controls the maximum tokens allowed in the generated response. The InferenceConfiguration reference states that the default is the model’s maximum allowed value. Leaving it unset is therefore not a product-specific short-answer policy. Select an explicit limit for the feature and validate it against that model’s supported maximum.
A concise-output instruction describes the desired answer, while maxTokens imposes an upper bound. Use both when the feature needs brevity, but do not assume a bound guarantees a complete answer. When generation reaches the limit, Converse.stopReason can be max_tokens. Treat that stop reason as a signal to inspect completeness and response validity. A truncated explanation or incomplete structured response must not pass solely because its usage was low.
On bedrock-runtime, token quota management initially deducts total input tokens plus max_tokens, the generation ceiling; consumed quota is adjusted for actual generation and the model-specific output burndown rate. An oversized inferenceConfig.maxTokens can reduce concurrency and throughput even when answers finish short. Right-size it from observed completions to bound generated output and upfront tokens-per-minute (TPM) reservation; the ceiling is not itself the final billed output count.
maxTokens does not prune the input. If the expensive portion is a long retrieved context, reducing the response ceiling addresses only the output side. Conversely, pruning input does not cap a model that continues generating a long answer. With model-specific InvokeModel bodies, parameter names depend on the model; do not mechanically copy the Converse field into every native request format.
Key distinctions and trade-offs
Pruning versus compression: Pruning removes information. Compression preserves selected information in a shorter form. Both require tests, but their failure modes differ: missing evidence versus altered meaning.
Input budget versus output ceiling: CountTokens measures the supplied input for a model. maxTokens limits generated output. Neither proves answer quality, and neither is a substitute for the other.
Shorter prompt versus lower request cost: A compression invocation, retry, or escalation can offset the initial saving. Compare total tokens and accepted results across the complete request.
Brevity versus truncation: A naturally concise complete response is successful. A max_tokens stop can indicate an incomplete response even when the request itself succeeded.
Architecture / decision scenario
A support assistant sends repeated instructions, full conversation history, and several overlapping policy passages to Bedrock. It also produces lengthy answers to simple questions. The team evaluates three controls separately so that a regression can be traced to its cause.
First, the context assembler removes duplicate passages and irrelevant history while retaining policy exclusions and source identity. It counts the final target-model input through the supported token-counting path. Second, it compresses repeated instructions and summarizes older dialogue into a versioned record of constraints and open questions. The quality set includes cases where an exclusion changes the correct answer.
Third, the gateway sets a feature-specific inferenceConfig.maxTokens and requests a concise answer. It records stopReason, response validity, and final token usage. Cases that end at max_tokens without required evidence fail the acceptance gate; the team adjusts the ceiling or requested answer scope rather than counting them as savings.
The winning policy lowers input and output usage while keeping accepted-answer coverage and total request cost within the agreed targets. Compression work is included in that cost comparison, and dependency-sensitive context handling remains governed by 1.3.9.
Common exam traps
- Treating
maxTokensas an input-context limit. - Assuming the model-maximum ceiling is harmless to quota if answers finish short.
- Using character count or another model’s tokenizer as the final budget check.
- Dropping source qualifications to reach an arbitrary compression ratio.
- Assuming
CountTokensis supported for every model on the same endpoint. - Counting a truncated response as a successful cost reduction.
- Ignoring the inference cost of a model-generated compression summary.
What to remember
Prune irrelevant information, compress useful information carefully, and cap generated output explicitly. Count the assembled input for the target model, then observe actual response usage. Evaluate preservation of meaning and completion, especially when stopReason is max_tokens. Optimize total cost per accepted request rather than the smallest possible prompt.
Knowledge check
Retrieved context dominates a support assistant’s input tokens: several passages repeat the same policy, and older dialogue is irrelevant. One policy exception decides many answers. The team lowered maxTokens, but input usage did not change. Which change reduces input cost without changing what the answers are based on?
maxTokens further and ask for a more concise answermaxTokens bounds generated output and never prunes input. Pruning selects information by its contribution to the task, so duplicates and irrelevant history go first while the deciding exception and its source identity stay. A fixed cut or a shortest-passage rule can drop exactly the qualification that changes the answer.A context assembler checks its input budget by counting characters, then applying a ratio measured on a different model. Requests to the target model sometimes exceed the budget. Which check fixes this?
CountTokens for the target modelmaxTokens so the response fits inside the remaining budgetCountTokens returns the input count the target model would be charged for, without charge. A character ratio or another model’s tokenizer only approximates it. Support still varies by model and endpoint, so confirm the selected model’s counting path.A team proposes summarizing conversation history with an extra model call. Most conversations in this feature are a single short turn. Which comparison should decide whether to adopt the summary?
After the team lowers the output ceiling, several structured responses end with stopReason set to max_tokens and miss required fields. How should those responses count in the savings comparison?
max_tokens stop that leaves the response invalid fails the acceptance gate, and the follow-up work to repair it offsets the apparent saving; adjust the ceiling or the requested scope instead.A feature’s answers finish well under 500 tokens, but maxTokens is left at the model maximum. The service hits its tokens-per-minute quota at lower traffic than expected. What explains it?
max_tokensmax_tokens value on every requestmax_tokens value makes the model generate longer answersbedrock-runtime , quota management deducts total input tokens plus max_tokens when the request starts and adjusts afterward for actual output. An oversized ceiling reserves capacity the answers never use, which lowers concurrency; it is not the billed output count.