AWSAWS Certified Generative AI Developer - Professional2026-104.2 Optimize application performance
Shorten the wait for useful output
Completed by the knowledge check
Learning objective
Reduce perceived latency using response streaming, precomputation, and parallel independent steps.
For an architect improving an interactive assistant, the outcome is a design that delivers useful output sooner while preserving freshness, authorization, and completion semantics. See 2.4.4 for stream transport contracts and 4.2.1 for latency attribution.
Core idea
Perceived latency depends on when the reader can use the response, not only when the server finishes. Streaming changes delivery timing. Precomputation moves reusable work out of the request path. Parallel execution overlaps independent prerequisites. These techniques solve different parts of the wait.
Choose each technique from a measured dependency graph. Streaming cannot hide a long retrieval stage that must finish before generation starts. Precomputing an answer helps only if its inputs and authorization remain valid. Parallelizing dependent steps can produce the wrong result sooner.
Retain two acceptance measures: time to first useful output and time to an accepted complete result. A quick progress message might reassure the reader, but it is not evidence that the answer itself arrives sooner.
How it works
Stream content without confusing completion
Amazon Bedrock provides ConverseStream and InvokeModelWithResponseStream for streaming inference. In the Converse stream, contentBlockDelta provides content changes, messageStop provides stopReason, and metadata provides usage and performance data. The application must distinguish these events rather than interpret every event as another text token.
Forward eligible content incrementally through the delivery contract described in 2.4.4. Measure first content at the model caller and at the client. If the model emits early but the client displays everything at the end, an intermediary or application buffer is defeating the benefit.
Streaming lets a reader begin reading before generation completes. It does not establish that the model finishes faster, and it does not eliminate retrieval prerequisites. Monitor Bedrock TimeToFirstToken alongside InvocationLatency to distinguish a faster initial response from a faster complete invocation.
Decide which content can be released before final validation. A drafting assistant can show provisional prose; a workflow consuming validated structured data might need the whole result. Do not feed an incomplete streamed object into a consequential tool action merely because its first fields look valid. See 3.1.2 for safety enforcement on streaming paths.
Give the interface explicit in-progress, complete, and failed outcomes. If the stream fails after partial output, mark that content incomplete. Retrying without a replacement rule can duplicate visible text or make two attempts appear to be one accepted response; see 2.4.5 for retry boundaries.
Precompute work with a reuse contract
Precomputation means doing predictable work before an interactive request needs it. A document assistant can generate document embeddings during ingestion rather than embed the full corpus for each question. Amazon Titan Text Embeddings produces vectors, and the vector database performs similarity search. AWS distinguishes low-latency invocation for retrieval from throughput-optimized batch jobs for indexing.
Keep document embedding work separate from query embedding work. The corpus exists before the request; the user’s exact question might not. A precomputed document vector does not remove the need to represent a previously unseen query.
Attach a reuse contract to each precomputed artifact: source revision, transformation version, embedding model, and authorized scope. This is an application design requirement, not an automatic guarantee from the embedding API. When a source or embedding contract changes, rebuild the affected artifacts before treating them as equivalent; see 1.4.11.
You can also prepare deterministic context fragments or frequently requested summaries. Reuse them only when their source and entitlement conditions still hold. A global summary that includes restricted evidence cannot become safe merely by being computed earlier.
For a long, repeated prompt prefix, Bedrock prompt caching reuses processed prompt work on supported models and can reduce response latency; see 4.1.6 for the caching choices.
Evaluate preparation cost against actual reuse. Work done for thousands of hypothetical requests that never arrive can increase cost without improving observed latency. Record reuse frequency, invalidation frequency, and the remaining request-path work before expanding the precomputation set.
Overlap independent prerequisites
AWS Step Functions implements separate concurrent branches with a Parallel state and its Branches array. Each branch defines its own States and StartAt. Step Functions executes those branches as concurrently as possible and waits until every branch terminates before following the state’s Next.
A retrieval lookup and a product-information lookup can overlap when each has validated input and neither needs the other’s result. The approximate serial wait is their sum; the approximate parallel wait is the longest branch plus orchestration and merge overhead. These are dependency estimates, not AWS latency guarantees.
Authorization must still precede any retrieval operation that depends on its result. Do not move permission-filtered retrieval into a branch that races the permission lookup. Identify the shared trusted inputs first, then fork work that is independent under those inputs.
Parallel execution also changes failures and demand. An unhandled branch error or a transition to Fail fails the entire Parallel state and stops its branches. AWS notes that invoked Lambda functions continue running after that failure. Do not assume workflow cancellation retracts work or side effects. Activity workers can use SendTaskHeartbeat failures to detect a stopped branch and stop their work. For Lambda fallback cleanup, use a Wait state so cleanup happens after the function finishes. Under shared trusted inputs the two lookups fork, and the state follows Next only after every branch terminates.
Use the documented Retry and Catch fields to define recovery at the appropriate boundary. Preserve a request deadline and decide which branches are mandatory. Starting more calls at once can increase concurrent downstream pressure; bounded concurrency and retry policy remain necessary.
Key distinctions and trade-offs
Streaming versus shorter inference: Streaming changes when output becomes available. Measure completion separately to establish an inference-speed improvement.
Precomputation versus unconditional reuse: Preparing an artifact early does not prove its freshness or authorization for a later caller.
Parallelism versus speculation: Independent branches use known valid inputs. Speculation starts work before a dependency is resolved and needs an additional correctness and waste policy.
First content versus accepted result: Provisional text can be useful to a reader while still being unsuitable for automated consumption.
Each technique has a different cost boundary. Streaming keeps a response path active; precomputation spends resources before demand is known; parallel branches create overlapping downstream work. Choose from observed user benefit and complete operating cost.
Architecture / decision scenario
A product-support assistant waits for document retrieval, product status, and then a model answer. Its document corpus changes daily, but every request repeats preparation work and performs the two lookups sequentially.
The team moves document embedding to ingestion using Titan Text Embeddings and versions the vectors with the source revision. It still embeds each unseen query on demand. After validating the user’s entitlements, it uses Step Functions Parallel branches for authorized retrieval and an independent product-status lookup.
The generation step starts after both required results arrive and streams through ConverseStream. The interface displays provisional text, then marks completion from the stream outcome. If the status branch fails, Catch selects a defined failure response rather than generating from missing mandatory evidence.
Measurements separate first displayed text, complete accepted answer, and precomputation reuse. The team retains all three changes only if the response benefit survives load testing and freshness checks.
Common exam traps
- Streaming before retrieval and assuming the model has the required evidence.
- Reporting an early progress message as model
TimeToFirstToken. - Precomputing corpus vectors but claiming query embedding disappears.
- Reusing artifacts after source or entitlement changes without validation.
- Parallelizing retrieval with the authorization result it requires.
- Assuming a failed
Parallelstate stops an already invoked Lambda function.
What to remember
- Stream eligible content; preserve explicit completion and failure.
- Move predictable reusable work out of the interactive path.
- Define freshness, model compatibility, and authorization for reuse.
- Parallelize independent branches after shared prerequisites.
- Measure useful output and accepted completion under representative load.
Knowledge check
A drafting assistant calls ConverseStream, and traces show the first contentBlockDelta arriving early. Users still see the whole answer appear at once when generation ends. Which change improves time to first useful output?
InvocationLatencyA document assistant embeds its whole corpus on every question. The corpus changes daily, and user questions are rarely repeated. Which precomputation plan removes the repeated work?
A request needs an entitlement check, a permission-filtered retrieval that uses the check’s result, and a product-status lookup that needs only the product ID. Which Step Functions plan is valid?
Parallel branchesParallel branchesParallel branchesA Parallel state fails because one branch raises an unhandled error. Another branch had already invoked a Lambda function that writes a draft record. What should the fallback assume?
Parallel state fails, Step Functions stops its branches, but invoked Lambda functions continue to run and cannot be stopped. A fallback that cleans up must wait, for example with a Wait state, rather than assume cancellation retracted the work.