AWSAWS Certified Generative AI Developer - Professional2026-102.2 Implement model deployment strategies

Buy capacity for the measured demand

Completed by the knowledge check

Learning objective

Select Bedrock on-demand inference, Provisioned Throughput, or SageMaker endpoints for workload capacity and cost.

Core idea

Choose a capacity model after you describe the work that arrives. Request count alone is insufficient for a foundation model: a short classification and a long generated report consume different amounts of inference capacity. Record input length, output length, arrival rate, burst duration, and the latency the caller can tolerate.

Amazon Bedrock on-demand inference gives you a managed invocation path without purchasing model units. Provisioned Throughput purchases a defined throughput level for a supported model. Amazon SageMaker AI hosting gives you a deployment whose model artifacts, container, and compute configuration you select. These choices allocate different responsibilities as well as different costs.

The decision is not “managed or unmanaged.” SageMaker AI also manages hosting infrastructure. The useful distinction is whether you consume a managed model resource or configure the serving deployment. See 1.1.3 for the operating-responsibility comparison; this lesson concerns the capacity purchase behind that decision.

How it works

Start with on-demand consumption

For a base model, InvokeModel.modelId identifies the model ID or ARN. An inference profile is another supported resource identifier; its processing-location implications belong to 1.2.4. The request still has to meet the selected model’s input contract and account quotas.

For text-model cost estimates, use the selected model’s input-token and output-token prices on the Bedrock pricing page. Do not assume all models or modalities share one billing unit. Build the estimate from representative request sizes, including generated output, rather than multiplying request count by an invented flat rate.

On-demand access does not mean unlimited capacity. The Bedrock service-quota table separates model-specific invocation and token-rate quotas, with Region-specific values and adjustment rules. Check the entry for the actual model and inference route. ThrottlingException means the request exceeded account quotas; increasing the caller’s concurrency does not remove that boundary.

On-demand is a useful starting point when demand is uncertain, intermittently idle, or still being measured. That is an architecture choice, not a promise that it always costs less. Long outputs and repeated calls can make a lightly used application expensive. Measure cost per accepted result, as described in 1.2.2.

InvokeModel.serviceTier accepts priority, default, flex, or reserved. Standard (default) is the default tier. Priority offers the fastest responses at a premium over standard on-demand pricing; Flex discounts workloads that tolerate longer processing. Priority, Standard, and Flex share the model’s on-demand quota. Check the model card for tier support before choosing one.

Reserved capacity is separate from that quota and reserves input and output tokens per minute, with minimums of 100,000 input TPM and 10,000 output TPM. Reservations have one- or three-month terms and fixed TPM-based pricing billed monthly. Arrange them through your AWS account team; billing continues until the reservation is deleted with your account manager’s help. It is not a Provisioned Throughput MU purchase.

Purchase a defined throughput level

Bedrock Provisioned Throughput is billed hourly. You specify a number of model units (MUs), and each MU defines input-token and output-token throughput per minute for the selected model. AWS directs you to your account manager for the model-specific MU throughput and price details. An MU is not a universal number of requests per second.

The documented commitment choices are no commitment, one month, and six months. A longer commitment discounts the hourly price, but the committed resource cannot be deleted before its term ends. Billing continues until you delete the Provisioned Throughput. Treat deletion and commitment expiry as separate events when estimating the service’s lifecycle cost.

Verify model and Region eligibility before comparing prices. For an API purchase, AWS requires a contextual variant of the foundation model as the model ID; check the supported-model table rather than substituting its on-demand ID. Invoke the purchased resource by passing its provisioned-model ARN as modelId to InvokeModel, or to a supported conversational or streaming operation. Purchasing capacity while leaving the application pointed at the base model does not use that purchase.

The request’s modelId and serviceTier decide which capacity it draws on.

Size MUs from both sides of the workload. If outputs are the limiting resource, spare input-token capacity does not prove the deployment can handle the arrival rate. Obtain the model-specific capacity figures, test the intended prompt and output distributions, and leave headroom for the bursts you actually need to serve.

Configure hosted capacity

SageMaker AI offers real-time, serverless, asynchronous, and batch deployment choices. Those modes divide by request size and latency; see 2.2.2. For an instance-based deployment, Application Auto Scaling policies adjust hosted capacity as workload changes. Model size alone does not establish generated-token throughput; see 2.2.5.

The documented scale-to-zero path for real-time endpoints requires inference components. Set the variant’s ManagedInstanceScaling.MinInstanceCount and the component’s minimum scalable capacity to zero. To recover capacity, attach an Application Auto Scaling step scaling policy to a CloudWatch alarm on NoCapacityInvocationFailures. Invocations fail while instances are absent and during the several minutes needed to provision capacity. Choose this idle-cost reduction only when your caller can accept that recovery behavior.

When comparing hosted inference with Bedrock, include the deployment’s active compute, idle capacity where applicable, storage, and operational work. Avoid treating the price of one instance as the complete cost of an application. The cost comparison must use the same accepted-output rate and latency target across candidates.

A hosted endpoint is appropriate when you need a separately served model or container behavior that the chosen managed invocation path does not provide. It can also be a capacity choice when measurements support it. Owning an endpoint does not automatically make its capacity cheaper or more predictable: you still have to load-test and operate the selected deployment.

Key distinctions and trade-offs

Usage charges versus hourly capacity: On-demand text inference prices the model’s token consumption. Provisioned Throughput bills purchased MUs by the hour, including periods when their capacity is unused. Compare both over the same demand trace.

Service tier versus purchased capacity: Priority and Flex change on-demand cost and priority. Reserved purchases TPM capacity outside the on-demand quota; Provisioned Throughput purchases model-specific MUs.

Quota versus purchased throughput: An account’s on-demand quota is an admission boundary. An MU describes provisioned input and output throughput. Neither is an application concurrency setting.

Model availability versus capacity availability: A model can support an invocation path without supporting Provisioned Throughput in your intended Region. Eligibility comes before cost optimization.

A commitment can improve unit economics for sustained demand but increases the cost of changing your mind. Keeping a bursty workload on demand can avoid idle provisioned charges but still requires quota planning. A hosted model adds serving control and a deployment to maintain.

Architecture / decision scenario

A support platform has three workloads. A pilot assistant receives unpredictable requests with long idle periods. A production summarizer receives a sustained stream of similarly sized reports. A specialist team needs a model served through its own approved container.

The architect starts the pilot on Bedrock on demand, checks its model-specific quotas, and records input and output consumption. For the summarizer, the team obtains MU throughput figures and compares hourly Provisioned Throughput against the measured on-demand demand trace. It buys capacity only after confirming model/Region support and proving that both input and output rates fit; the application then invokes the provisioned ARN.

The team evaluates supported Flex pricing for delay-tolerant summaries and compares Reserved TPM capacity separately from MUs. The specialist model goes to SageMaker AI because its serving contract requires that deployment. The team chooses its inference mode separately and includes idle compute and operating work in the comparison. If sustained traffic falls, it reviews the provisioned commitment and deletion policy rather than assuming unused capacity stops billing.

Common exam traps

Treating on-demand access as unlimited throughput.
Confusing Reserved tier TPM reservations with Provisioned Throughput MUs.
Sizing MUs by request count without input and output token rates.
Assuming every Bedrock model supports Provisioned Throughput in every Region.
Buying a provisioned resource while invoking the base-model ID.
Assuming a commitment’s end automatically deletes the billed resource.
Comparing a single hosted instance’s price with complete managed inference cost.

What to remember

Measure arrival patterns and input/output demand before choosing capacity.
Check the exact model, Region, resource identifier, and quotas.
Provisioned Throughput purchases model-specific MUs billed hourly.
Use the provisioned-model ARN to consume provisioned capacity.
Compare complete cost at the same quality, latency, and accepted-output target.