AWSAWS Certified Generative AI Developer - Professional2026-104.1 Implement cost optimization and resource efficiency strategies

Choose the right level of cache reuse

Completed by the knowledge check

Learning objective

Distinguish Bedrock prompt caching, deterministic exact-match response caching, and semantic response caching.

Core idea

A cache saves work only at the layer whose result it reuses. Amazon Bedrock prompt caching reuses processing of eligible prompt prefixes while the model still generates a response. An application response cache returns an answer produced earlier and can avoid the generation call entirely. An exact response cache requires the same effective request; a semantic response cache accepts a different request judged equivalent enough for reuse.

For an architect choosing a cost optimization, the deciding question is therefore what must remain equal: a prompt prefix, a complete request contract, or the meaning of a request within an approved context. These are three different equivalence rules. Each creates different failure modes and needs separate evidence of savings and correctness.

How it works

Reuse prompt processing

Bedrock documents implicit and explicit prompt caching, with support varying by model and API. Implicit caching attempts to reuse eligible prefixes without request breakpoints. Explicit caching marks a reusable prefix with model-specific controls. For supported Converse models, the marker is a cachePoint content block with type: "default"; available placement depends on the model.

Keep static instructions and reference content before variable questions. A change to the prefix can cause a miss even when the topic remains the same. In Converse, checkpoint processing follows tools, then system, then messages; changing an earlier section invalidates reuse for later sections. Adding a checkpoint is an eligibility request, not a guarantee of a hit.

Eligibility also depends on token count and lifetime. For example, the Bedrock table lists Amazon Nova Pro (amazon.nova-pro-v1:0) with a 1K-token minimum prefix, up to four checkpoints, a five-minute time to live (TTL), and checkpoints in system and messages. Nova prompt caching has a 20K-token maximum. These are model-specific limits, not a contract for every Bedrock model. A prefix below the minimum can still produce a successful inference without being cached.

The documented prompt cache TTL resets after a successful hit. For models supporting a one-hour TTL, Converse accepts "ttl": "1h" in cachePoint; omit ttl for five-minute-only models because including it can return a ValidationException. Bedrock limits prompt caching to on-demand inference endpoints and excludes the batch inference API. Check the chosen model and API rather than assuming support transfers from one invocation path to another.

Converse reports usage.cacheReadInputTokens and usage.cacheWriteInputTokens, which show actual reuse and cache creation. Generation still occurs, so usage.outputTokens remains relevant. Cache-read and cache-write input pricing differs from ordinary input pricing and varies by model; repeated writes without enough reuse can undermine the expected benefit. Cost attribution belongs to 4.1.1.

Reuse an identical request’s answer

An exact response cache sits in application code before inference. Build a deterministic key from the complete effective request, look up the stored answer, and return it only if its validity gates pass. Amazon DynamoDB can implement this path with GetItem for lookup and PutItem for a validated query-response record. Those operations store and retrieve application data; they do not automatically understand model prompts.

“Deterministic” describes identity construction and lookup. It does not establish that a fresh model invocation would return identical text. A request with sampling, tools, or changing evidence can yield different results even when the displayed user question repeats. Reuse is appropriate only if the product permits returning the previously accepted answer under the same effective contract.

The effective request includes the instructions, conversation state, selected model configuration, source revisions, authorization scope, and output requirements that affect the answer. Caching only the latest text silently converts contextual questions into context-free ones. See 4.1.7 for key construction and invalidation, and 4.1.9 for access boundaries.

Reuse an equivalent question’s answer

A semantic response cache embeds an incoming query and searches stored query embeddings. AWS documents a semantic cache built on ElastiCache for Valkey vector search with Amazon Bedrock embeddings, which compares the candidate’s similarity score with an application threshold.

A qualifying candidate lets the application skip response generation. It does not necessarily skip the embedding call or candidate validation. The cache stores query-response pairs, so a high query similarity says that two questions are close in embedding space; it does not prove that the stored answer satisfies the new question.

Scope semantic matching to stable, authorized contexts. A product identifier, version, locale, or entitlement difference can change the required answer despite similar phrasing. AWS recommends application-context filters and treats highly dynamic responses as poor caching candidates. Threshold calibration and answer relevance checks belong to 4.1.8.

Key distinctions and trade-offs

The following comparison identifies the work and assumptions each cache changes:

Cache	Reuse identity	Generation on a hit	Main correctness concern
Bedrock prompt cache	Eligible matching prefix	Continues	Prefix eligibility and measured token reuse
Exact response cache	Complete deterministic request identity	Can be skipped	Missing key inputs or stale dependencies
Semantic response cache	Similar query within a compatible scope	Can be skipped	Related questions requiring different answers

Prompt caching is useful when users ask many different questions about the same long reference. Exact response caching fits repeated requests whose answer contract is stable. Semantic caching can cover paraphrases, but introduces false acceptance as a distinct quality risk.

These mechanisms can coexist. An application can try an exact lookup, then an eligible semantic lookup, then invoke Bedrock with a cacheable prefix on a miss. This is an application design, not an automatic Bedrock pipeline. Each stage must retain authorization and freshness checks, and its cost must include the work done before a hit is accepted. A hit at either response cache returns an answer produced earlier, and only a miss at both reaches Bedrock, where the model still generates a response.

Architecture / decision scenario

A software-support assistant receives three traffic patterns: different questions against a common manual, repeated identical product questions, and paraphrases of installation requests.

For different questions, the team places the stable manual and instructions before the question and uses a supported Bedrock prompt checkpoint. It confirms reuse from cacheReadInputTokens; it still budgets for generation and output tokens.

For repeated product questions, it uses a DynamoDB exact response cache keyed by the effective request and manual revision. GetItem returns a candidate only for that identity, and the application checks expiration before returning it. A revised manual makes the previous identity ineligible.

For installation paraphrases, an ElastiCache semantic lookup operates only within the same product and version. A candidate must pass the calibrated threshold and answer relevance checks. Questions about removing the product bypass an installation answer even if the wording is similar. All response-cache misses go to inference; prefix reuse can still reduce that miss path’s input processing.

Common exam traps

Claiming that a Bedrock prompt cache hit returns a previously generated answer.
Treating an explicit checkpoint as proof of a hit or universal model support.
Interpreting deterministic lookup as deterministic model generation.
Building an exact response key from the last user message alone.
Accepting a semantically related query without validating its answer and context.
Counting only avoided generation calls while ignoring embeddings, lookup, and validation.

What to remember

Prompt caching reuses input processing; response caching reuses an answer. Exact matching is a complete request-identity decision, while semantic matching is a calibrated approximation within a compatible scope. Measure each layer separately and return cached answers only after authorization, freshness, and correctness gates pass.