AWSAWS Certified Generative AI Developer - Professional2026-104.1 Implement cost optimization and resource efficiency strategies
Attribute inference spend to accepted work
Completed by the knowledge check
Learning objective
Track and attribute input/output tokens and inference cost per request, feature, and tenant.
Core idea
An inference bill describes purchased processing, while a product request describes work a customer wanted completed. Those boundaries differ when one request invokes retrieval, several models, tools, retries, or an escalation. Use AWS attribution mechanisms for usage and billed cost, then retain application attempt and acceptance records to explain completed-task economics.
For an architect allocating Amazon Bedrock spend, the decision is which evidence supports each accounting level. Per-request metadata tagging gives invocation-log detail. Application inference profiles and IAM principal attribution supply aggregated billed allocation. Application records add workflow acceptance and attempt relationships that service accounting does not establish.
Measure cost per accepted result as well as cost per invocation. A cheaper call that repeatedly produces rejected output can increase the cost of completing the user’s task. Define acceptance independently of token count, then retain both attempted work and successful outcomes in your reports.
How it works
Capture usage at the invocation boundary
The Bedrock Converse response contains usage.inputTokens, usage.outputTokens, and usage.totalTokens. Its metrics.latencyMs records invocation latency, and stopReason records why generation stopped. For ConverseStream, usage arrives in the stream’s metadata event. Do not calculate final usage from the number of text chunks displayed to the user: chunks are a delivery representation, not a token ledger.
AWS’s per-request metadata tagging attaches dimensions to individual bedrock-runtime calls. Set requestMetadata in Converse and ConverseStream, or the X-Amzn-Bedrock-Request-Metadata header in InvokeModel and InvokeModelWithResponseStream. For manually signed requests, include that header in SigV4 SignedHeaders. Metadata is retained only when model invocation logging is enabled in the call’s Region; it is not supported on bedrock-mantle.
Invocation logs carry a top-level requestMetadata field alongside input, output, cache-read, and cache-write token counts. Group counts by tenant, feature, or request identifier with CloudWatch Logs Insights, or use Athena over S3 logs, then apply the appropriate rates. Bedrock does not enforce metadata presence: set trusted dimensions in a shared client or gateway. Keep sensitive data out of tags; see 3.2.8 and 3.2.9 for logging controls.
Retain logical request and attempt identifiers, model target, completion status, and accepted-result status in your application ledger. Correlate these records with log metadata to combine AWS usage attribution with application outcomes. A second invocation receives a separate attempt record even when the customer sees one answer.
A stream that disconnects before your application receives final metadata leaves an accounting gap. Mark usage as unknown rather than inventing zero tokens. Preserve the attempt identity for later reconciliation against available service evidence. Do not automatically repeat generation merely to obtain a cleaner accounting record; retry behavior belongs to the execution policy in 2.4.5.
Convert usage into a scoped estimate
For an uncached text invocation billed by input and output tokens, estimate input charge and output charge separately using the applicable rates and their published units. AWS pricing varies by model and modality; the pricing page also distinguishes inference options. Record the rate’s effective context alongside your calculation so that a future price change does not rewrite historical estimates.
Do not multiply totalTokens by one arbitrary rate. Input and output pricing can differ. The usage object also exposes cacheReadInputTokens and cacheWriteInputTokens; when caching applies, retain these categories and use the model’s documented pricing rules. Treat cache accounting as a distinct pricing case rather than applying the uncached formula indiscriminately. Cache design is outside this lesson’s request-attribution boundary.
Label the result an estimate until you reconcile it with billing data. Token estimates do not cover every charge in a complete application. Keep retrieval, guardrail, tool, storage, and orchestration spend in separate categories when calculating end-to-end feature economics. Likewise, an hourly Provisioned Throughput purchase needs an explicit shared-capacity allocation policy; it is not a per-call on-demand token bill. See 4.1.5 for utilization decisions.
Reconcile aggregates with resource allocation
For the bedrock-runtime endpoint, CloudWatch publishes InputTokenCount, OutputTokenCount, Invocations, and InvocationLatency under AWS/Bedrock. These metrics provide aggregate sanity checks. Bedrock publishes runtime metrics with and without the ModelId dimension; failures before the target model is identified appear without that dimension. A model-filtered dashboard therefore cannot prove that every request was accounted for.
Application inference profiles attach resource tags to on-demand usage. Supply the profile ARN as modelId, and activate its tags as cost allocation tags in the Billing and Cost Management console. They then support Cost Explorer and Cost and Usage Reports (CUR), including CUR 2.0, at per-usage-type-per-day grain, not per request. A profile’s allocation excludes calls that bypass it. For Responses and Chat Completions on bedrock-mantle, use Projects; application inference profiles do not support those APIs.
IAM principal attribution automatically captures the calling IAM user or role. Principal tags and AWS STS session tags add dimensions; activate the cost allocation tags for billing visibility. Cost Explorer and CUR 2.0 provide per-usage-type-per-day allocation. For identity detail in CUR 2.0, configure an export that includes the caller identity ARN. Session tags distinguish sessions of a shared role, but are not per-call metadata.
Choose the mechanism’s grain deliberately: shared feature profiles can supply feature billing allocation while per-call tenant metadata supplies tenant detail in logs. Reconcile log estimates with CUR at model and usage-type grain. Neither classic CUR nor CUR 2.0 contains a per-request identifier in its line items; a requestId join to individual bill rows cannot supply per-request reconciliation. Keep shared-capacity allocations explicit. One call therefore feeds two evidence paths, which meet only when log estimates are reconciled with CUR at model and usage-type grain.
Key distinctions and trade-offs
Preflight count versus observed usage: CountTokens estimates model-specific input usage before inference. It is useful for admission and budgeting, but it cannot report generated output or subsequent attempts. The completed response is your per-call usage evidence.
Request versus attempt: A request can contain several chargeable processing steps. Summing only the final successful attempt understates feature cost and hides expensive fallback behavior.
Request detail versus billed allocation: Request metadata provides per-call token detail in logs, not Cost Explorer or CUR tags. Profiles, Projects, and principal tags provide aggregated billing allocation.
Measured versus allocated cost: Token usage is a measurement. Dividing a shared hourly capacity purchase among tenants is an accounting policy. Present both honestly rather than making an allocation appear to be an AWS per-request charge.
Architecture / decision scenario
A document assistant serves two tenants through a summarization feature and a question-answering feature. The billing dashboard reports rising spend, although customer request volume is flat. Both features use application inference profiles with activated cost allocation tags. The gateway supplies their ARNs as modelId and sets trusted tenant and feature keys in requestMetadata; invocation logging is enabled in the call Region.
CloudWatch Logs Insights groups token counts by tenant and feature metadata. It reveals longer inputs for one tenant and additional question-answering invocations. The application ledger joins attempts to the logical request and marks validation failures and accepted results. Missing final stream metadata remains a reconciliation category rather than zero usage.
CloudWatch token metrics corroborate overall demand. Profile tags explain aggregated feature billing; request metadata explains the tenant split. The team reconciles model and usage-type totals against CUR, then evaluates context reduction and quality-gate changes by total cost per accepted result, including failed attempts.
Common exam traps
- Treating
totalTokensas sufficient for pricing with different input and output rates. - Assuming CloudWatch
ModelIdmetrics automatically include feature and tenant identity. - Ignoring per-request metadata tagging, or expecting its tags in Cost Explorer or CUR.
- Joining invocation
requestIdto CUR line items that have no per-request identifier. - Counting only the answer delivered to the user while ignoring earlier model attempts.
- Recording zero usage when a stream ends before final usage metadata arrives.
- Presenting an on-demand estimate as the exact allocation of an hourly capacity purchase.
What to remember
Capture usage per invocation attempt and join it to trusted application identity. Aggregate attempts into requests before calculating feature and tenant economics. Use request metadata for log-based detail and activated profile or principal tags for aggregated billing allocation. Preserve the distinction between estimates, measured tokens, billed spend, and allocated overhead.
Knowledge check
A request’s first invocation fails output validation, so the gateway invokes a larger model and that answer is accepted. The feature report must show what it cost to complete the user’s task. What should the usage ledger record?
CountTokens estimates input before inference, so it cannot report output or later attempts.A model’s output tokens cost more than its input tokens. Each invocation log record carries input and output token counts. How should the team estimate the uncached charge per invocation?
totalTokens misprices any call whose mix differs from the assumption. Stream chunks are a delivery representation, not a token count. Price each measured category at its own rate and label the result an estimate until it is reconciled with billing data.A platform team needs monthly spend per feature in Cost Explorer and a token breakdown per tenant for individual calls. Each feature calls Bedrock through its own path. Which combination meets both needs?
ModelId dimensionModelId metrics carry no tenant identity, and session tags identify a session rather than each call.A response stream disconnects before the application receives the final metadata event that carries usage. How should the ledger record that attempt?