AWSAWS Certified Generative AI Developer - Professional2026-104.1 Implement cost optimization and resource efficiency strategies
Assign model tiers from measured evidence
Completed by the knowledge check
Learning objective
Route queries to model tiers using measured quality, capability, latency, and price.
Core idea
A model tier is an application decision, not a guarantee implied by model size or marketing position. A low-cost tier is useful only for requests it can complete correctly within the required deadline. Select eligible models first, measure them on the feature’s workload, and then route to the least expensive option that passes the quality and latency gates.
For an architect designing a Bedrock routing policy, the crucial separation is capability eligibility versus economic preference. Missing a required modality, response interface, or context capacity disqualifies a model. A lower price cannot compensate for that absence. Among eligible models, measured quality and latency establish whether a cheaper route is acceptable.
Routing machinery and model contracts are covered in 1.2.3 and 2.4.7. This lesson focuses on the evidence that assigns requests to tiers. A routing service can execute a choice, but your product’s acceptance criteria must establish whether the choice makes economic sense.
How it works
Gate candidates by required capability
Describe request classes in terms of required behavior: input modality, response contract, context needs, and interaction requirements. Keep business constraints, such as approved processing locations, in the eligibility gate as well. These are requirements to validate against each candidate’s documented support, not assumptions about an entire model family.
Converse provides a common Bedrock invocation contract for supported models, with the target in modelId. A shared API does not mean every target supports every possible feature. Preserve explicit eligibility records for the exact model and invocation path. If a request cannot meet a required contract on a candidate, exclude it before comparing token prices.
Use request classes that the gateway can identify reliably. A trusted feature identifier is often a stronger signal than asking a model to decide whether a prompt is important. If an application classifier chooses among eligible classes, include its error rate, latency, and inference cost in the routing evaluation. An incorrect classification can send a consequential task to a tier that was never approved for it.
Benchmark the complete route
Run a representative evaluation set through each eligible candidate. The benchmark design in 1.2.2 establishes the operating envelope; the routing decision adds a question: which request classes meet their gates on each model? Measure required facts, response validity, and accepted-result rate for each class rather than one average score over unrelated work.
Include realistic context lengths, languages, tool-result shapes, and difficult cases. Do not benchmark a compact prompt and then route production requests that include long retrieval evidence. Preserve prompt and model identity with results so that the tier assignment is reproducible after a release.
For non-streaming Converse, metrics.latencyMs supplies invocation latency. For streaming routes, CloudWatch TimeToFirstToken applies to ConverseStream and InvokeModelWithResponseStream, while InvocationLatency measures time until the last token is received. These are different experience measures. A route can start quickly but finish too late to meet the feature’s delivery requirement.
Measure gateway-to-accepted-result latency as an application metric too. A model’s timing does not include every surrounding operation or subsequent attempt. Evaluate tail latency under the expected load, not only a quiet demonstration. Define your percentile targets as product requirements; do not present an illustrative target as an AWS service guarantee.
Compare economics after acceptance
AWS pricing is specific to the model, modality, and selected service tier; see 2.2.1 for tier mechanics. Combine the applicable price with observed usage.inputTokens and usage.outputTokens; for caching or other pricing cases, use their documented categories. See 4.1.1 for the complete attribution boundary. Price per token alone does not tell you the price of completing a task.
A tier that needs longer answers, repeated clarification, or a second model attempt can cost more per accepted result than a higher-priced direct route. Account for classification work and escalation frequency. An economical routing policy minimizes completed-task cost within quality and deadline constraints, rather than maximizing the proportion of requests sent to the cheapest model.
For application-controlled routing, the policy resolves the eligible tier to the appropriate Converse.modelId. Record the route decision and policy version alongside observed usage and acceptance. When workload changes, rerun the benchmark and revise the mapping deliberately; do not let a transient price or latency observation silently override a capability gate.
Distinguish managed routing from your measured policy
Amazon Bedrock intelligent prompt routing predicts response quality and routes among supported models within a model family. CreatePromptRouter names models, a fallbackModel, and routingCriteria.responseQualityDifference. The fallback is the baseline used when the routing criterion is not met; it is not your application validator’s decision after inspecting a generated answer.
AWS documents two important limits: intelligent prompt routing is optimized only for English prompts, and it cannot adjust routing decisions or responses using application-specific performance data. Its quality prediction therefore does not implement a custom policy over your quality labels, observed latency, and business acceptance rules.
Use a managed router when its supported scope and predicted-quality behavior fit the request class, then evaluate the complete route against your requirements. Use application routing when the decision must enforce your own measured capability, latency, or quality thresholds. An application cascade that validates one answer before escalating is a separate mechanism; see 2.2.9.
Key distinctions and trade-offs
Capability versus quality: Capability asks whether the request contract is supported. Quality asks how well an eligible candidate performs the task. Both gates are necessary.
Token price versus accepted-result cost: A cheaper token rate can be offset by longer outputs and repeated attempts. Compare the cost of the workflow outcome.
First token versus completed answer: TimeToFirstToken describes response start. InvocationLatency and application completion measurements address later boundaries.
Predicted quality versus application evidence: responseQualityDifference controls a managed router’s criterion. It is not a switch that imports your task labels or production latency targets into the router.
Architecture / decision scenario
A customer-support gateway handles short policy lookups, long document analysis, and multilingual questions. Two eligible text models are candidates for the routine lookup tier; document analysis also needs the supported context and input contract. The team verifies capabilities before measuring savings.
Its evaluation shows that the lower-priced candidate passes English lookup quality and completion-latency gates. It fails the required acceptance rate on complex exclusions, so that request class uses the stronger direct route. Multilingual questions have their own measured mapping; the team does not assume an English-optimized intelligent prompt router supplies equivalent routing quality for them.
For each route, the gateway records modelId, usage, metrics.latencyMs, and the application’s acceptance result. Streaming features separately track TimeToFirstToken. The team compares cost per accepted result, including classification and any escalation. A managed router is tested as an alternative for the eligible English class, but its responseQualityDifference setting remains distinct from the application’s quality and latency gates.
When production requests become longer, the team remeasures the class rather than preserving the tier assignment because the token rate stayed low. The evidence changes the policy, while capability restrictions remain mandatory.
Common exam traps
- Assuming the smallest or cheapest model is always the economical route.
- Treating a common
ConverseAPI as universal feature support. - Using average model latency as proof of tail end-to-end deadline compliance.
- Treating a router fallback as post-generation application validation.
- Assuming intelligent prompt routing learns from your application’s acceptance labels.
- Applying an English-optimized routing prediction to every language without evaluation.
What to remember
Gate by capability, then measure quality, latency, and completed-task cost by request class. Keep the chosen model and policy version observable. Managed quality prediction and application-defined routing evidence serve different decisions. Reevaluate tier assignments when prompts, models, or workload composition change.
Knowledge check
A request class requires image input. The cheapest candidate model accepts text only, but it scored best per dollar on the team’s text summarization benchmark. What should the routing policy do with that candidate for this class?
Converse interface does not add support it lacks. Benchmark scores from a different request class say nothing about this one.A low-priced model passes the quality gate for routine lookups, but a third of its answers need a clarification turn or an escalation to a stronger model. How should the team decide whether it remains the lookup tier?
A gateway serves English and Spanish questions, and routing must follow the team’s own acceptance labels and latency targets. A colleague proposes Amazon Bedrock intelligent prompt routing with a tuned responseQualityDifference. Why does this not meet the requirement on its own?
responseQualityDifference as a probability that answers are correctresponseQualityDifference sets when to switch away from the fallback baseline; it does not import your labels, so a measured application policy needs its own routing logic.A streaming feature’s requirement is that the complete answer reaches the user within eight seconds. One candidate shows a low TimeToFirstToken. Which measurement decides whether it meets the requirement?
TimeToFirstToken at the expected load percentileTimeToFirstToken across a sample of short promptsTimeToFirstToken describes when a stream starts; InvocationLatency runs until the last token, and the application’s own gateway-to-accepted-result timing adds surrounding work and retries. A route can start fast and still finish after the delivery deadline.