AWSAWS Certified Generative AI Developer - Professional2026-104.3 Implement monitoring systems for GenAI applications
Bound invocation logs by diagnostic purpose
Completed by the knowledge check
Learning objective
Configure Bedrock Model Invocation Logs to diagnose model behavior, including log destinations and retention policies.
Core idea
Invocation logs help you inspect what a model received and returned. They are a diagnostic data store containing potentially sensitive payloads, not a harmless extension of aggregate runtime metrics.
Amazon Bedrock model invocation logging is disabled by default. Its configuration applies to supported invocations in an AWS account and Region, and its destinations must be in that same account and Region. Choose what data to collect, where to deliver it, how to detect missing delivery, and when each stored copy expires.
Treat logging enablement and data retention as separate controls. Stopping future collection does not establish that previously delivered CloudWatch events or S3 objects have been removed.
How it works
Confirm the endpoint and collection boundary
Model invocation logging supports calls through the bedrock-runtime endpoint. The documented operations include Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream. AWS also documents support for the OpenAI-compatible Responses and Chat Completions APIs on that endpoint. Calls through bedrock-mantle are not captured by this logging feature.
This boundary matters when a dashboard shows successful inference but no payload record. Check the endpoint before interpreting missing logs as delivery failure. Do not assume one logging configuration covers every Bedrock API surface.
Use PutModelInvocationLoggingConfiguration to configure collection, GetModelInvocationLoggingConfiguration to inspect it, and DeleteModelInvocationLoggingConfiguration to remove the configuration. The loggingConfig structure separates destination settings from data-delivery switches.
The switches include textDataDeliveryEnabled, imageDataDeliveryEnabled, embeddingDataDeliveryEnabled, audioDataDeliveryEnabled, and videoDataDeliveryEnabled. Enable the data needed for an approved diagnostic purpose rather than selecting every modality by habit. These switches describe collection settings; they do not make an unsupported inference endpoint observable.
Choose the destination from the payload
The configuration supports CloudWatch Logs, Amazon S3, or both. cloudWatchConfig specifies logGroupName and roleArn; s3Config supplies an S3 destination. The optional cloudWatchConfig.largeDataDeliveryS3Config provides an S3 location for large data associated with CloudWatch delivery.
Both destinations can hold invocation metadata and input and output JSON bodies up to 100 KB inline. Binary data or JSON bodies larger than 100 KB require S3 objects. S3 invocation files contain batched records in gzipped JSON; large or binary payloads are separate objects referenced from the record. CloudWatch events contain the corresponding JSON invocation records and use the configured S3 large-data location when needed.
For Converse, image or document data passed to the model is logged in S3 when S3 delivery and image logging are enabled. A CloudWatch-only design is therefore not a complete payload archive for a diagnostic workflow that depends on those inputs.
Choose CloudWatch Logs when operators need searchable event records through Logs Insights. The CloudWatch Model Invocations view also uses these logs to display model inputs and outputs; see 4.3.1. Choose S3 when the approved investigation requires larger payloads or retained files. Using both can serve different consumers, but it creates multiple retention and access boundaries. Account for every copy before describing the collection as time-bounded.
Grant delivery permission and inspect records
The CloudWatch delivery role trusts bedrock.amazonaws.com through sts:AssumeRole. The documented role policy permits logs:CreateLogStream and logs:PutLogEvents for the destination stream. The documented trust policy constrains aws:SourceAccount and aws:SourceArn.
For S3 delivery, the documented bucket policy permits s3:PutObject by the Bedrock service principal with source-account and source-ARN conditions. If the destination uses customer-managed KMS encryption, configure the required key permission as well. Read the destination policy against the selected account and Region; a correctly named bucket does not prove that delivery is authorized.
A log record includes schemaType, schemaVersion, timestamp, requestId, operation, modelId, and identity.arn. Its input.inputTokenCount and output.outputTokenCount describe usage, while inline bodies or S3 references provide payload evidence. Optional requestMetadata is caller supplied; the other documented fields are populated by Bedrock.
Correlate these records with the application request mapping from 4.3.1. Diagnose a wrong answer by comparing the captured input, configuration, retrieved evidence, and output. A log shows the exchanged content; it does not prove why a model chose a particular answer or whether the answer is true.
Enforce retention in each destination
For CloudWatch Logs, PutRetentionPolicy sets logGroupName and retentionInDays. Choose a supported value, such as 7 or 30 days, according to the approved policy. The account/Region configuration specifies one CloudWatch log group through cloudWatchConfig.logGroupName, so its invocation records share that group’s retention period. Retention can differ between account/Region configurations, but one log group cannot assign different periods to individual data classes. S3 Lifecycle rules can assign retention by prefix, including separate log and large-data prefixes.
CloudWatch deletion is asynchronous. AWS states that deletion typically takes up to 72 hours after events reach their retention setting and can take longer in rare situations. Do not promise immediate erasure at the retention boundary.
For S3, use Lifecycle Expiration rules on the invocation-log and large-data prefixes. In a versioning-enabled bucket, expiration of a current object creates a delete marker and makes that object version noncurrent. Add NoncurrentVersionExpiration where permanent removal of noncurrent versions is required. See 3.2.10 for deletion across retained copies and constraints that can prevent expiration. Each stored copy expires only under its own destination’s control.
Finally, monitor AWS/Bedrock delivery metrics. ModelInvocationLogsCloudWatchDeliveryFailure, ModelInvocationLogsS3DeliveryFailure, and ModelInvocationLargeDataS3DeliveryFailure identify failed delivery attempts. Inspect the matching success metrics too. Test a representative invocation and confirm that its record and any referenced payload arrive.
Key distinctions and trade-offs
Invocation logs versus aggregate metrics: Logs support payload diagnosis. Runtime metrics support numerical health monitoring without requiring full prompt capture.
Configuration removal versus stored-data deletion: Removing the Bedrock configuration controls logging. CloudWatch retention and S3 Lifecycle control destination data independently.
Inline records versus payload objects: A searchable event may contain only a reference to a large payload. Retaining the event while expiring the object changes what an investigation can reconstruct.
Longer retention improves the chance of reproducing a delayed complaint, but it increases the period of exposure for source documents and customer text. Set retention from an explicit investigation need and apply the privacy controls from 3.2.8 and 3.2.9.
Architecture / decision scenario
A document assistant produces disputed summaries several days after a release. The team needs short-lived searchable records plus the original document inputs for approved review.
It enables text and the relevant image/document collection, configures CloudWatch with an S3 large-data destination, and verifies the delivery role and bucket policy. A test request appears with its requestId and an S3 reference. Delivery-failure metrics stay clear during the test.
The team sets a seven-day CloudWatch retention policy and S3 expiration rules for both log and payload prefixes, including noncurrent versions where applicable. It documents asynchronous deletion rather than promising erasure at midnight on day seven. Requests sent through bedrock-mantle require a separate application diagnostic path.
Common exam traps
- Assuming invocation logging is enabled by default.
- Choosing a destination in a different account or Region.
- Treating CloudWatch-only delivery as sufficient for every binary or large payload.
- Deleting the logging configuration as the sole retention mechanism.
- Expiring current S3 versions while retaining all noncurrent versions.
- Interpreting an absent payload record without checking endpoint support and delivery metrics.
What to remember
- Scope logging to the supported endpoint, account, Region, and needed data types.
- Use S3 for binary data and JSON bodies larger than 100 KB.
- Verify permissions and delivery with a representative request.
- Enforce CloudWatch and S3 retention independently.
- Preserve correlation without treating a payload log as an explanation of correctness.
Knowledge check
Invocation logging is enabled in an account and Region. Calls from one service through bedrock-runtime produce log records, but calls from a second service through bedrock-mantle produce none, although they succeed. What is the most likely cause?
logs:PutLogEvents on the destination log grouptextDataDeliveryEnabled to falsebedrock-mantlebedrock-runtime endpoint only. A permission or data-delivery problem would also affect the bedrock-runtime calls that are being logged, so the difference between the two services points at the endpoint boundary.A document assistant calls Converse with images and long documents, and investigators need the original inputs. Logging currently delivers only to a CloudWatch log group. Which change closes the gap?
embeddingDataDeliveryEnabled in the logging configurationS3 delivery is configured, but a representative test request leaves no record, and ModelInvocationLogsS3DeliveryFailure rises. Which check comes first?
bedrock-runtime endpoints3:PutObject for the Bedrock service principal under the source conditions, plus any customer-managed KMS key permission. An unsupported endpoint produces no attempt at all, and the CloudWatch role does not govern S3 delivery.Policy requires invocation payloads to be unrecoverable after 30 days. The versioning-enabled bucket has a 30-day Expiration rule on the log and data prefixes, yet an audit recovers 90-day-old payloads. Which change meets the policy?
NoncurrentVersionExpiration to the rules for both prefixesretentionInDays to 30 on the CloudWatch log groupimageDataDeliveryEnabled for future invocationsExpiration adds a delete marker and leaves the old version noncurrent and recoverable; NoncurrentVersionExpiration removes those versions. Removing the logging configuration or changing collection only affects future data, and CloudWatch retention governs a different copy.