AWSAWS Certified Generative AI Developer - Professional2026-104.1 Implement cost optimization and resource efficiency strategies

Match purchased capacity to usage trends

Completed by the knowledge check

Learning objective

Optimize Provisioned Throughput, batching, and resource utilization from actual usage trends.

Core idea

Purchased capacity becomes economical when useful demand occupies it consistently. A busy demonstration is not evidence of sustained utilization, and a high daily average can hide both idle hours and short overloaded intervals. Use actual input and output token trends, request mix, latency, and successful outcomes before committing to throughput or changing how work is scheduled.

For an architect operating Amazon Bedrock, Provisioned Throughput is a capacity purchase with a lifecycle and model-specific constraints. Batching is a scheduling or processing choice. Resource utilization is a measurement of useful work against available capacity. These decisions interact, but they are not interchangeable controls.

The objective does not imply one provisioning mode or one universal batch mechanism. AWS documentation distinguishes Provisioned Throughput by Model Units from provisioning by Tokens. AWS batch inference is a separate asynchronous job service and does not support provisioned models. Keep those boundaries explicit when optimizing from observed demand.

How it works

Measure demand in the units capacity serves

For Model Unit provisioning, an MU supplies model-specific throughput expressed as input tokens processed and output tokens generated within one minute. AWS directs you to your account manager for the throughput specification and pricing per MU. Do not assume one MU has the same token capacity across models or that a request-per-second figure describes both input and output pressure.

Use AWS/Bedrock runtime metrics such as InputTokenCount, OutputTokenCount, Invocations, InvocationLatency, and InvocationThrottles, alongside the attempt ledger in 4.1.1. Group data by the inference target and comparable time windows. Retain the application’s record of which requests used the provisioned resource so that mixed on-demand and provisioned traffic does not obscure the purchase’s useful load.

Compute application-level input and output utilization ratios separately against the verified capacity specification. A request mix with long answers can saturate output capacity while input capacity appears lightly used. Another mix can reverse the bottleneck. Request count alone misses that difference.

Inspect repeated peaks, quiet intervals, and workload composition over a representative operating cycle. Include scheduled imports, business-hour traffic, and changed context lengths. Daily totals help with economics; shorter windows expose demand bursts and deadline pressure. Correlate rising latency or throttles with changes in token mix before concluding that more capacity is required.

Do not use EstimatedTPMQuotaUsage as a proxy for MU utilization; see 4.1.4 for its runtime quota caveat.

Evaluate the purchase and its lifecycle

Provisioned Throughput by Model Units is billed hourly. Its cost depends on model, number of units, and commitment duration. CreateProvisionedModelThroughput names modelId and modelUnits; its commitmentDuration supports OneMonth and SixMonths, while omitting that field requests no commitment where supported. Model and commitment eligibility must be checked for the target purchase.

Compare the fixed hourly spend with the cost of serving the same accepted workload through eligible alternatives. Include idle purchased hours and the application’s required headroom. A workload can justify dedicated capacity for operating requirements even when it is not the lowest-cost route, but that is a deliberate decision rather than a claim that fixed capacity is always cheaper.

The lifecycle matters after purchase. AWS states that one-month and six-month commitments prevent deletion until their terms end, and billing continues until the Provisioned Throughput is deleted. AWS also documents automatic renewal at each commitment term, maintaining token configurations. Assign an owner to the applicable retirement or renewal action rather than assuming term expiry stops spend.

To use the purchased resource, supply its provisioned model ARN as modelId to a supported runtime invocation operation. Purchasing capacity while continuing to target another inference resource does not demonstrate that the application is using the purchase. Validate the routing target as part of utilization review.

Respect the provisioning mode when changing capacity

AWS’s modification guide says Model Unit purchases permit edits to name, tags, and limited custom-model associations. UpdateProvisionedModelThroughput exposes desiredProvisionedModelName and desiredModelId; it does not expose a modelUnits resize field. Do not describe this API as automatic scaling or instruct operators to reduce units through it.

For a changed Model Unit requirement, plan a purchase and routing transition appropriate to the supported lifecycle, account limits, and existing commitment. A replacement can overlap the existing resource and add cost, so evaluate the timing explicitly. Changing a name or tag is not a capacity reduction.

The same modification guide states that provisioning by Tokens permits modification of input and output tokens per minute. That behavior belongs to that provisioning mode; it does not extend the Model Unit update API. Verify the supported model and interface before relying on such a change. For Tokens mode, Cancel auto renew prevents the next renewal: capacity stays in service through the term, and the full current-term fee remains due. Cancellation cannot be reversed and blocks further modifications for the rest of that term. This lifecycle control does not remove the restriction on early deletion of committed MU purchases.

Use batching only within the right boundary

Application-level grouping can collect compatible work and schedule it into quieter intervals when deadlines permit. This is your scheduling policy, not proof that a Bedrock runtime request contains a documented multi-request batch interface. Each actual model call still needs its own supported request format, attribution, and completion record.

Amazon Bedrock batch inference instead uses CreateModelInvocationJob and S3 input and output. It is not supported for provisioned models. Moving offline work to that service changes the execution path rather than filling unused Provisioned Throughput. See 4.1.4 for compatibility, deadline, and job-state decisions.

Use the distinction to evaluate real savings. Scheduling flexible work onto otherwise idle purchased capacity can improve useful utilization if capacity and deadlines allow. Moving that work to discounted batch jobs can reduce variable inference cost, but it can leave a committed resource more idle. Compare the complete bill and accepted outcomes under both policies, including overlapping capacity and recovery work.

Key distinctions and trade-offs

Peak capacity versus sustained utilization: Peak demand helps establish required headroom. Sustained useful demand determines how much fixed spend is occupied over time.

Input versus output capacity: Track both directions against their specifications. One low ratio does not prove the resource has enough capacity in the other direction.

Model Units versus Tokens: Provisioning modes have different modification rules. A token-rate change is not evidence that UpdateProvisionedModelThroughput can resize MUs.

Grouping work versus batch inference: Scheduling calls together is an application policy. Bedrock batch inference is a separate job path that excludes provisioned models.

Term expiry versus billing stop: Track renewal and retirement explicitly. Tokens-mode Cancel auto renew preserves the paid current term while preventing renewal; committed MU purchases cannot be deleted early.

Architecture / decision scenario

An enterprise assistant uses a provisioned model ARN for steady daytime requests. A separate summarization queue produces flexible evening work. Operations sees low overnight input utilization but daytime output peaks, so increasing MUs from daily request counts would be poorly justified.

The team compares observed InputTokenCount, OutputTokenCount, latency, and accepted work with its verified MU specification. It finds that long daytime answers drive the limiting direction. Output reduction is evaluated through 4.1.2, and flexible evening calls are scheduled only within their deadline slack and the purchased resource’s remaining capacity.

The team also evaluates the S3-backed batch job path for summaries. Because it cannot use provisioned models, that option is priced as a separate route. Savings are compared against the still-running hourly purchase rather than counted as improved utilization of it.

A smaller future MU purchase could match the revised workload, but the existing commitment restricts retirement. The team assigns an owner to the MU transition and eventual deletion; a Tokens-mode retirement would instead consider Cancel auto renew before expiry. It records the required routing change and retains the Model Unit update restrictions. It measures the next operating cycle before choosing the next commitment.

Common exam traps

Equating high request count with balanced input and output utilization.
Assuming an MU has identical throughput across foundation models.
Treating EstimatedTPMQuotaUsage as exact provisioned-capacity accounting.
Claiming UpdateProvisionedModelThroughput resizes Model Units.
Sending Bedrock batch jobs to provisioned models to fill idle capacity.
Assuming the end of a commitment automatically stops hourly billing.

What to remember

Measure useful input and output demand over realistic operating cycles. Compare fixed spend with accepted-work economics and required headroom. Respect the purchase’s modification and retirement rules, and target its ARN deliberately. Optimize scheduling and batch inference as separate paths, with the complete bill as the economic boundary.