AWSAWS Certified Generative AI Developer - Professional2026-102.2 Implement model deployment strategies
Bound Lambda callers before invoking models
Completed by the knowledge check
Learning objective
Invoke models on demand from Lambda while handling cold starts, timeout, and concurrency limits.
Core idea
A Lambda function can be the caller of an inference service without being the host of the foundation model. In that design, the function validates the request, prepares the model payload, invokes the selected resource, and handles the response. Model weights and generation remain behind the inference service boundary.
Separate four budgets: function initialization, function execution time, invocation payload size, and downstream model capacity. Improving one does not expand the others. A warm Lambda function can still wait on a slow model, and an available Lambda concurrency slot can still issue a request that Bedrock throttles.
This lesson uses standard Lambda function invocations as its baseline. AWS also documents Lambda Managed Instances and separate execution options; do not apply a timeout from one invocation mode to another without checking its scope.
How it works
Keep initialization out of every request
AWS recommends initializing SDK clients outside the handler so an execution environment can reuse them. Use persistent connections through the runtime’s keep-alive mechanism, while accounting for Lambda purging idle connections. Recreating a client and connection for every inference request adds work unrelated to generation.
Reuse is an optimization, not durable application storage. Keep request-specific data inside the handler and avoid retaining user data or sensitive events in the shared execution environment. Warm environments must not leak one caller’s prompt or retrieved evidence into another caller’s request.
Lambda provisioned concurrency allocates pre-initialized execution environments and incurs additional charges. It can reduce initialization latency for interactive callers. It does not warm the separately hosted model, increase Bedrock throughput, or remove the time spent generating tokens. If a downstream serverless model also cold-starts, diagnose that boundary separately; see 2.2.2.
Lambda SnapStart instead initializes a function when you publish a version and resumes execution environments from a cached snapshot. It supports Java 11+, Python 3.12+, and .NET 8+ managed runtimes, but not Node.js or OS-only runtimes. SnapStart cannot combine with provisioned concurrency. Use provisioned concurrency when SnapStart cannot meet strict cold-start requirements. Validate connections after restore, and generate unique request state after initialization rather than reusing snapshotted values.
Reserved concurrency is a different control. It reserves concurrency for the function and caps the function at that amount, without pre-initializing the environments. Use PutFunctionConcurrency to configure it. Do not select reserved concurrency when the requirement is specifically to reduce cold starts.
Fit the complete call within the deadline
The standard Lambda function timeout maximum is 900 seconds, or 15 minutes. The quota page documents an exception for Lambda Managed Instances: asynchronous and eligible event-source-mapping invocations can allow up to 5400 seconds, or 90 minutes, excluding Amazon MQ and Amazon DocumentDB mappings. That exception does not extend a standard synchronous function call.
Set the function timeout from a load-tested request path, not from the maximum the service permits. The path includes validation, serialization, the SDK request, model processing, response handling, and any allowed retry. Reserve time to return a controlled outcome before the function deadline expires.
Configure the SDK’s connection/read timeouts and retry policy to fit the same budget. A socket timeout that exceeds the function’s remaining execution time cannot produce useful recovery. A retry budget that ignores the model’s generation time can spend the whole deadline on another attempt. See 1.2.5 for bounded retry and failover decisions.
Use InvokeModel when the selected Bedrock model requires a native request body, or a supported conversational operation when appropriate. The model still has its own parameter and response contract. Calling from Lambda does not change those requirements; see 1.3.6 and 1.3.7.
For work that does not fit the caller’s deadline, change the response contract. A short function can submit an asynchronous inference job and return a job identifier rather than wait for completion. That is an architecture choice from 2.2.2, not a reason to keep increasing the synchronous function timeout.
Bound payloads before creating the model request
Standard synchronous Lambda invocations allow 6 MB for each request and response. Asynchronous invocation payloads are limited to 1 MB. Synchronous response streaming permits up to 200 MB per streamed response; that does not increase the request limit. These are Lambda invocation boundaries, not the model service’s payload limits. A payload permitted by Bedrock can still be too large to enter or leave the function through the chosen invocation mode.
Lambda memory allocation ranges from 128 MB to 10,240 MB, and CPU allocation grows with configured memory. Increasing memory can help local parsing or serialization, but it does not assign more compute to the remote model. Benchmark the caller separately from generation.
The /tmp storage range is 512 MB to 10,240 MB. It can support reusable static assets, but temporary disk is not a durable result store. For large job inputs, use the S3-backed inference path where applicable rather than trying to carry the entire job through a small Lambda event.
Match concurrency to downstream work
The documented default regional concurrent-executions quota is 1000, with reduced quotas possible for new accounts. For standard functions, the concurrency scaling limit is 1000 execution environments per function every 10 seconds. These numbers describe Lambda capacity, not model invocation quotas.
Estimate in-flight demand from arrival rate multiplied by average request duration, then validate bursts and tail behavior. As an illustrative calculation, 20 requests per second with a five-second handler duration require about 100 concurrent executions in steady state. If model latency doubles, the same arrival rate occupies about twice as many slots.
Reserved concurrency can bound the number of active callers and protect downstream resources. It is not a requests-per-minute or tokens-per-minute limiter. Shorter calls can issue more requests through the same concurrency cap, and longer prompts can consume more model capacity per request. Keep model-specific quota admission and Lambda concurrency as separate controls.
Key distinctions and trade-offs
Caller cold start versus model latency: Lambda initialization belongs to the caller. Inference belongs to the selected resource. Measure both before buying warm caller capacity.
Reserved versus provisioned concurrency: Reserved concurrency reserves and caps slots. Provisioned concurrency pre-initializes environments. Neither purchases Bedrock model units.
SnapStart versus provisioned concurrency: SnapStart restores initialized state from a snapshot. Provisioned concurrency keeps environments ready. They are alternatives, and runtime eligibility constrains the choice.
Function timeout versus request timeout: The function deadline bounds all handler work. An SDK timeout bounds one network operation. Configure the latter and retries inside the former.
Monitor ConcurrentExecutions, Throttles, and Duration in CloudWatch. Duration excludes cold-start time, so it is not the whole user-observed latency. ProvisionedConcurrencySpilloverInvocations identifies requests that use standard concurrency when provisioned concurrency is exhausted. Record model-service errors separately so Lambda throttling is not confused with Bedrock ThrottlingException.
Architecture / decision scenario
An application invokes Bedrock through Lambda. After idle periods, first requests are slow; under sustained load, requests also fail despite low local CPU usage.
The team moves SDK initialization outside the handler and enables connection reuse. It checks the complete user latency because Duration omits cold starts, then provisions warm Lambda environments only if initialization prevents the latency target. It sets reserved concurrency to bound active callers and monitors ConcurrentExecutions and Throttles.
The sustained-load failures are Bedrock ThrottlingException, so adding Lambda concurrency would worsen the pressure. The team applies model-quota admission and a bounded retry policy within the function deadline. Large report jobs move to the asynchronous submission path; interactive requests retain a measured synchronous budget. Each change addresses the failing boundary rather than treating all delays as cold starts.
Common exam traps
- Confusing a model caller with a model host.
- Using reserved concurrency to eliminate initialization latency.
- Assuming warm Lambda environments also warm downstream model capacity.
- Treating Lambda concurrency as a token-rate limiter.
- Applying the Managed Instances timeout exception to standard synchronous calls.
- Assuming
Durationincludes cold-start time or that Lambda and model payload limits match.
What to remember
- Separate initialization, execution, payload, and model-capacity budgets.
- Reuse SDK clients and connections without retaining caller data.
- Fit request timeouts and retries inside the function deadline.
- Reserve concurrency to cap callers; provision concurrency to reduce initialization.
- Diagnose Lambda throttling separately from downstream quota failures.
Knowledge check
A Node.js Lambda function calls Bedrock for an interactive assistant. After idle periods, function initialization makes first requests miss the latency target; model latency itself is within budget. Which change reduces the initialization delay?
Under sustained load, a Lambda function that calls Bedrock shows low CPU, no Lambda Throttles, and a rising count of Bedrock ThrottlingException errors on long prompts. What should the team change?
A Lambda function has a 30-second timeout. Its Bedrock SDK client uses a 60-second read timeout and up to three retries. On slow responses the function is killed mid-wait and returns no controlled error. What fixes it?
An API client sends 9 MB documents to a Lambda function invoked synchronously, which forwards them to a model whose input limit is larger. The invocations fail before the handler runs. What design works?
A team’s dashboard shows a low p99 for the Lambda Duration metric, yet users report that the first request after an idle period is slow. What explains the gap?
Duration excludes cold-start time, so initialization does not appear in itDuration excludes Bedrock wait time, so model latency does not appear in itDuration is sampled, so slow first requests are dropped from itDuration counts only provisioned concurrency, so on-demand requests are missingDuration measures time the function code spends processing an event and does not include cold-start initialization, so it understates what a user sees on a cold request. Model calls made by the handler do count, which is why the gap points at initialization.