AWSAWS Certified Generative AI Developer - Professional2026-102.2 Implement model deployment strategies

Prove smaller models meet the task

Completed by the knowledge check

Learning objective

Deploy smaller task-specific pre-trained models from SageMaker JumpStart when they meet the quality bar at lower cost and latency

Core idea

For an architect selecting a production model, a narrow task can justify a narrow model. Classification, embedding, and extractive question answering need different outputs from open-ended generation. If a pre-trained model reliably supplies the required output, a larger general-purpose model might add expense and delay without improving the business result.

SageMaker JumpStart provides task-specific pre-trained models and deployment information. Its task-specific catalog includes text classification, text embedding, question answering, and other problem types. This gives you candidates to evaluate, not automatic proof of suitability. The catalog’s task label must match the application contract, and the model must pass your representative quality tests.

The objective contains three acceptance conditions: adequate quality, lower cost, and lower latency. Model size alone establishes none of them. Compare the deployed systems at the same traffic and acceptance criteria. A smaller endpoint that needs frequent correction or escalation can be more expensive per useful result than the larger alternative.

How it works

Match the model’s output to the decision

Define what the application consumes before searching the catalog. A support-ticket router needs an allowed category and a policy for ambiguous requests. A semantic-search pipeline needs embeddings compatible with its index. An extractive question-answering task needs an answer supported by the provided passage. These contracts are not interchangeable because each processes text.

JumpStart’s task-specific model detail page describes what the model does, expected inputs and outputs, and the data type needed for fine-tuning. Inspect those details before interpreting its responses. A pre-trained classifier’s original label set might not represent your business categories; a deployable model is not necessarily a usable model for that decision.

Use a candidate only when its pre-trained behavior matches the contract or a separately evaluated customization makes it fit. Do not hide a label mismatch with an informal mapping that has no validation. If fine-tuning is necessary, account for that change and its release evidence rather than describing the model as an untouched pre-trained solution.

The catalog helps you narrow the search. The SageMaker Python SDK’s list_jumpstart_models() returns available model IDs. A model ID names the candidate; it does not replace the evaluation, artifact, and interface records needed for a production release. Model approval belongs to 1.2.7.

Deploy a reproducible candidate

JumpStart deployment creates a SageMaker AI endpoint. Inspect the deployment configuration in Studio; default hardware is a starting point. Record the model, instance type, configuration, and test dataset for reproducibility.

AWS’s Python SDK foundation-model deployment documentation uses JumpStartConfig and ModelBuilder. JumpStartConfig(model_id=...) identifies the model; ModelBuilder.from_jumpstart_config(...) constructs its deployment builder. The documented path calls build() and deploy(), then invokes the endpoint. You can specify a model version on JumpStartConfig and an instance_type on ModelBuilder.

For those JumpStart deployments, instance_types.retrieve_default() identifies the default inference instance type for the model and version. instance_types.retrieve() lists supported types. Use that supported set to constrain an Inference Recommender benchmark. A default is a starting configuration, not a cost optimum or an assurance that the smallest instance sustains production concurrency.

Record the selected version and deployment configuration. Rerun affected tests after changing the model, preprocessing, or hardware.

The request and response contract is model-specific. AWS’s ModelBuilder example for FLAN-T5 uses an inputs key in a JSON body; that example does not establish the payload for a classifier or embedding model. For the selected model_id and model_version, discover formats with retrieve_serializer_options, retrieve_deserializer_options, content_types.retrieve_options, and accept_types.retrieve_options. The first two are the SDK documentation’s aliases for serializer and deserializer retrieve_options functions. Validate the payload schema as described in 1.3.8; format discovery does not establish field semantics.

Test quality before counting savings

AWS’s text-classification deployment guide compares DistilBERT Base Cased with BERT Base Uncased. It describes the smaller model as faster and more cost-effective, with potentially lower accuracy. This illustrates the trade-off, but it is not a numerical guarantee for your data or a substitute for testing both deployments.

Build a held-out evaluation set from the task’s real distribution. Include rare but important labels, ambiguous examples, domain terminology, long inputs, and requests outside the model’s intended task. For classification, examine per-class precision and recall as well as aggregate accuracy. A model can pass an overall score while consistently misrouting a small, consequential class.

Set acceptance criteria before measuring candidates. For a ticket router, the cost of sending a billing dispute to the wrong queue can differ from the cost of an unnecessary manual review. Choose thresholds and terminal outcomes from those consequences. Do not interpret a model score as an automatically calibrated probability of correctness.

SageMaker Inference Recommender automates load testing and tuning across instance configurations, considering instance count, container parameters, model optimizations, and maximum concurrency. CreateInferenceRecommendationsJob starts an instance-recommendation or load-test job. Supply a versioned Model Registry package or a SageMaker AI model; the latter is specified through ContainerConfig. For a registered package, SupportedRealtimeInferenceInstanceTypes constrains the instance search in a Default job to your supported set. Use the results to compare configuration cost and latency, not to infer task quality.

Compare client and tail latency under representative concurrency. Use ModelLatency and OverheadLatency to locate container versus platform time; see 2.2.6. A faster container does not guarantee a faster application.

Compute cost per accepted result over the same workload window. Include provisioned endpoint capacity, idle time, retries, preprocessing, human correction, and any larger-model escalation. A lower per-request compute estimate can lose when acceptance falls or endpoint capacity remains underused. Inference mode and capacity decisions are covered in 2.2.1 and 2.2.2.

Key distinctions and trade-offs

Task-specific versus general-purpose: A narrow model can provide exactly the required decision. It does not inherit the broader conversational capabilities of a generative model. Keep the narrow service’s supported inputs explicit.

Pre-trained versus task-ready: Pre-trained weights make a model available for inference. Your labels, domain, language, and error costs determine whether it is ready for this application.

Smaller versus cheaper: Fewer model resources can reduce serving requirements, but total cost also depends on demand, endpoint configuration, acceptance, and downstream work.

Lower model latency versus lower user latency: Container execution is only part of the path. Require the complete application to meet its response target.

Use the smaller candidate when the evidence supports all three acceptance conditions. Retain the larger candidate when it provides necessary quality that the smaller deployment cannot deliver within the allowed error cost.

Architecture / decision scenario

A support platform uses a general-purpose model to classify tickets and draft complex replies. The team separates those tasks before replacing anything. Classification becomes the candidate for a task-specific JumpStart deployment; open-ended reply drafting remains on the evaluated generative path.

The team compares a smaller classifier with a larger classification baseline using the same business labels and held-out tickets. It first checks that each candidate’s input and output contract matches the service. A candidate whose pre-trained labels cannot represent the queues fails before the load comparison.

For a candidate that passes, the team uses Inference Recommender to compare supported configurations, records model version and deployment settings, and measures client latency alongside the endpoint metrics. The smaller deployment passes the required per-class recall, reduces tail latency, and reduces cost per accepted classification at expected demand. Ambiguous cases follow the explicit manual-review policy.

The team deploys that candidate for classification only. If a rare safety-related category fails its quality floor, it keeps the stronger route for that class or rejects the replacement. Aggregate savings do not authorize silent degradation of the high-cost error category.

Common exam traps

Choosing a smaller model before checking its label or output contract.
Assuming every text task requires open-ended generation.
Treating a JumpStart default instance as the cheapest viable deployment.
Reusing one model’s JSON payload for every catalog model.
Reporting aggregate accuracy while a consequential class fails.
Counting compute savings without correction, escalation, or idle endpoint cost.

What to remember

JumpStart supplies deployable candidates and task information. You supply the acceptance contract. Match the model’s interface to the task, preserve the model and deployment configuration, and compare quality, complete latency, and cost per accepted result. Deploy the smaller model only when all required conditions pass.