AWSAWS Certified Generative AI Developer - Professional2026-102.5 Implement application integration patterns and development tools

Make generation outcomes explicit in APIs

Completed by the knowledge check

Learning objective

Design FM application APIs around streaming responses, token limits, model timeouts, and retry boundaries

Core idea

A foundation model (FM) application API must describe more than a request and a text response. It must tell callers whether generation started, whether visible output is provisional, why generation stopped, and whether another attempt is safe. Otherwise, the same network failure can look like an empty answer, a finished answer, or permission to generate twice.

Design the application contract around observable outcomes. Amazon Bedrock ConverseStream supplies content events, message termination, usage metadata, and error events. Your adapter translates them into stable application states without exposing model-specific payloads as the public interface. Streaming transport belongs to 2.4.4; here the decision is what each event promises to the caller.

The API also needs an admission boundary. Reject or reduce requests that cannot fit the chosen model’s input and output budget before starting a visible stream. Keep deadlines and attempt budgets explicit so a supported model capability does not become an unlimited promise to the application user.

How it works

Separate delivery from completion

ConverseStream returns an event stream containing messageStart, contentBlockStart, contentBlockDelta, contentBlockStop, messageStop, and metadata. Content blocks have indexes, so an adapter must preserve their relationship rather than concatenate every event into one undifferentiated string. A content delta is evidence of progress, not evidence that the response is complete.

Define application events such as started, delta, completed, and failed. These names are your contract, not Bedrock event names. Attach a request identifier and an attempt identifier to every event. The client can then associate an error with the correct visible answer and distinguish an explicit replacement from continuation of an interrupted answer.

Interpret messageStop.stopReason. end_turn indicates a different outcome from max_tokens, model_context_window_exceeded, tool_use, or guardrail_intervened. A model that requests a tool has not necessarily finished the application’s task. A response that stops at max_tokens might be readable but incomplete. Map model_context_window_exceeded to a separate context-exhausted outcome: reassess the assembled context and output allowance rather than treating it only as a truncated answer. Your completed event should state the application outcome and retain the underlying stop reason for diagnosis.

Collect metadata.usage.inputTokens, outputTokens, and totalTokens when supplied. A client disconnected before metadata arrived does not give you evidence that the invocation consumed zero tokens. Record missing usage as unknown, and distinguish measured usage from an admission estimate.

Admit requests within token budgets

Use CountTokens with the target modelId and the input corresponding to the request you intend to send. Tokenization is model-specific. For a conversation, the input.converse structure carries the messages and system content; for a native request, input.invokeModel carries the body. Count the assembled request, not only the user’s final sentence.

inferenceConfig.maxTokens limits generated output in Converse requests. It does not shorten the input conversation or establish a universal context-window size. Check the selected model’s supported limits and reserve an output allowance before admitting the input. Context assembly and history reduction are covered in 1.3.9; the API should expose their result as an explicit acceptance or rejection decision.

Keep application limits distinct from model limits. You might deliberately allow a smaller answer to meet a response-time target. Tell callers whether that policy rejects the request, reduces allowed output, or returns a truncated result. Silently changing these choices makes the same public request behave differently across model routes.

Bound deadlines at each layer

A service timeout, an SDK read timeout, and an application deadline measure different boundaries. ModelTimeoutException means processing exceeded the model timeout and has HTTP status 408. A local read timeout instead means the caller stopped waiting; it does not prove that the service performed no work.

AWS documents a 60-minute inference timeout for Anthropic Claude 3.7 Sonnet and Claude 4 models and recommends a read timeout of at least 3,600 seconds for those calls. That model-specific allowance is not a requirement to keep every interactive request open for an hour. Set the application’s response target from the user task, and choose a processing path that can satisfy it.

When the permitted work is longer than the interactive deadline, offer an asynchronous job contract rather than stretching an interactive promise. Acceptance should return a job identity; a separate status or result operation reports completion. Durable job handling belongs to 2.4.2. A browser timeout should not implicitly authorize another job.

Put retries before the visible boundary

Before exposing output, a bounded retry can be appropriate for a transient failure. After exposing output, restarting inference creates another generated answer, not a documented resume of the first answer. Require an explicit replacement policy rather than appending a retry’s text to the earlier fragment.

The AWS SDK settings AWS_RETRY_MODE and AWS_MAX_ATTEMPTS control retry behavior and attempts. Where your SDK version requires the opt-in, enable the reference’s updated retry behavior with AWS_NEW_RETRIES_2026=true; verify your installed SDK’s defaults. Configure and test your actual client, then include its attempts in the same application budget. See 2.4.5 for retry configuration and 1.2.5 for model fallback.

A failed attempt is retried only before output is exposed; after that, it needs an explicit replacement.

Retain the original request identity when a caller retries an application operation. If a result already exists, return that result according to your contract. If status is uncertain, report uncertainty or reconcile it. Generating again merely because a caller lost its connection can duplicate cost and any downstream actions.

Key distinctions and trade-offs

The following boundaries and outcomes guide the API decisions:

Boundary	What it controls	What it does not establish
`inferenceConfig.maxTokens`	Maximum generated output	Input fit or complete business meaning
`messageStop.stopReason`	Why model generation stopped	Whether the application task succeeded
`model_context_window_exceeded`	Context-exhausted generation outcome	Merely reaching the configured output cap
Application deadline	How long the caller’s operation may take	Whether timed-out downstream work never ran

Streaming improves perceived responsiveness but exposes partial results. Buffered delivery makes withholding incomplete answers easier, at the cost of waiting for the full result. Neither choice removes the need for terminal outcome states.

Architecture / decision scenario

A contract assistant streams a draft summary and can create a review request after approval. The public API assigns a request and attempt identity, counts the assembled input with CountTokens, and sets a task-specific output cap through inferenceConfig.maxTokens.

If Bedrock rejects the call before any delta reaches the client, the server can spend one remaining attempt within the deadline. If the stream fails after several paragraphs, the client marks that draft incomplete and offers an explicit replacement. It never appends a second attempt to the first.

A max_tokens stop produces a truncated outcome. An end_turn stop proceeds to application validation before the API reports success. Only a separately authorized operation creates the review request. Requests whose permitted generation cannot meet the interactive deadline use the durable job path, preserving the same request identity across status checks.

Common exam traps

Treating an HTTP connection opening as proof of a completed generation.
Using maxTokens as an input-token or context-window limit.
Treating tool_use as a finished business response.
Confusing an SDK read timeout with a model service timeout.
Restarting generation after visible output and calling it stream resumption.
Multiplying SDK, gateway, and application retries without a shared budget.

What to remember

Describe progress, termination, and application acceptance separately.
Count the assembled input for the selected model.
Preserve stopReason and measured usage in the outcome contract.
Set deadlines independently of the model’s maximum permitted duration.
Make replacement after partial output explicit and bounded.