Skip to lesson

AWSAWS Certified Generative AI Developer - Professional2026-101.3 Implement data validation and processing pipelines for FM consumption

Validate enrichment against the extracted evidence

Completed by the knowledge check

Learning objective

Normalize and enrich extracted content with Bedrock or Comprehend, then validate FM-transformed data.

Core idea

Extraction produces evidence; enrichment adds interpretation. Normalization can make content easier to consume, but it can also erase distinctions that matter: a date’s locale, an amount’s currency, a speaker’s qualification, or the source position of an entity. Keep the extracted original separate from the normalized view.

Choose Amazon Comprehend when the required enrichment matches a documented analysis operation, such as language or entity detection. Choose Amazon Bedrock when the task requires a foundation model to transform content into a domain-specific representation. In either case, the output is an inference result, not automatic proof that the transformed record is correct.

Validate at the point where derived data becomes usable by the next consumer. This lesson follows the extraction and provenance boundaries in 1.3.3 and 1.3.5; it focuses on what happens after extraction, rather than re-teaching modality acquisition.

How it works

Normalize without losing the source

Define a canonical application representation before invoking an enrichment service. For example, retain the source text, a normalized text view, its language decision, source identifiers, and the transformation version. These are application fields, not a universal AWS response schema.

Apply deterministic conversions where you have explicit rules. Normalize line endings or known formatting noise without guessing missing facts. Convert an ambiguous date only when its source context establishes the intended interpretation; otherwise preserve the ambiguity. A generative model’s plausible choice does not turn an unknown value into verified data.

If normalization changes character positions, keep a mapping to the original or identify the normalized view as a separate source version. Comprehend entity offsets refer to the text supplied for analysis. Using them against a differently normalized original can attach evidence to the wrong characters.

Retain measurement units and uncertainty alongside normalized values. Reject a transformation that silently chooses a currency or resolves an unsupported identifier. The quality-policy design belongs to 1.3.1; here the key is preserving enough evidence to apply that policy after enrichment.

Use Comprehend for a defined analysis

DetectDominantLanguage accepts UTF-8 Text with at least 20 characters and a maximum size of 100 KB. Its Languages array is ordered by descending Score; each entry includes LanguageCode and the confidence in that inference. Use those exact fields to implement your language-routing decision.

Do not send an isolated three-character fragment and assume that the operation’s response establishes its language reliably. Preserve document context or use a trusted source language under an explicit application policy. A language the service can detect is not automatically a language supported by every other Comprehend operation.

For standard entity enrichment, DetectEntities accepts UTF-8 Text, with a maximum size of 100 KB, and a LanguageCode from the supported operation values. Its Entities include Text, Type, Score, BeginOffset, and EndOffset. Preserve those outputs alongside the text version analyzed rather than retaining only a flattened list of names.

Use Entity.Score as confidence in detection, not confidence that the detected statement is true. An accurately detected organization name can occur in a false claim. Calibrate the application’s acceptance threshold against representative examples; do not turn a round number into a universal AWS quality requirement.

For a domain-specific recognizer, DetectEntities.EndpointArn selects a custom entity recognition endpoint. In that mode, Comprehend uses the custom model’s language and ignores a request LanguageCode. Supplying a different code does not translate the input or change the model’s language. Select a compatible recognizer before analysis.

When a text unit exceeds the synchronous operation’s size limit, prepare bounded units under an explicit splitting policy and preserve their source positions. Do not truncate mid-entity and then treat the absent entity as evidence that the original document contained none.

Use Bedrock for a constrained transformation

Give Bedrock a defined transformation task, source evidence, and an output contract. For example, ask it to derive a review record from extracted text, preserve unknown values, and associate claims with source identifiers. Converse request formatting is covered in 1.3.6; native payload validation is covered in 1.3.7.

On supported models, Bedrock structured outputs constrains responses to a user-defined JSON schema. For Converse, the mechanism is outputConfig.textFormat, with type: "json_schema" and the schema supplied through structure.jsonSchema.schema. The schema value is a serialized JSON schema string, not an arbitrary prose description.

Native InvokeModel fields differ: AWS documents output_config.format for Anthropic Claude and response_format for open-weight models. Validate the selected model’s feature support and invocation path rather than assuming that all three field names are interchangeable.

Alternatively, add strict: true to supported tool definitions to validate tool names and inputs against the tool input schema. For an extraction record emitted as a tool call, the same supported schema subset and application-validation requirements apply.

AWS supports a subset of JSON Schema Draft 2020-12. It supports basic types and enum, but does not support numerical constraints such as minimum, maximum, or multipleOf, string constraints such as minLength or maxLength, recursive schemas, or external references. additionalProperties must be false when used in that supported schema. Unsupported schema features cause an immediate HTTP 400 response.

Those boundaries leave work for your application. An integer field can conform to the generation schema while exceeding a business limit. A string can have the required type while containing an invented source identifier. Validate those conditions after generation.

There are also feature-combination restrictions: AWS documents structured outputs as incompatible with citations for Anthropic models. Enabling both produces HTTP 400. If you require that provider’s citation feature, choose a compatible output approach rather than combining incompatible settings. If you generate source identifiers as ordinary fields instead, validate them yourself; they are not equivalent to provider-generated citations.

Validate before releasing derived data

Use distinct gates for distinct claims. First, inspect invocation completion. With Converse, examine stopReason and the returned output.message before accepting an output as a complete transformation. Do not repair a capped response by treating its partial fields as a complete record.

Next, parse the result and enforce the consumer’s full schema, including constraints outside Bedrock’s supported generation subset. Then validate semantics: permitted values, arithmetic relationships, units, allowed identifiers, and consistency with the source. Finally, confirm that required claims resolve to retained evidence.

Implement these gates in application validation code; the schema-constrained generation mechanism does not perform every domain check. Produce an explicit validation result that records failed rules and source identifiers. Downstream consumers receive only accepted records, while rejected records remain available for correction or review under your workflow policy.

Key distinctions and trade-offs

Normalization versus invention: Normalization changes representation under known rules. Enrichment derives interpretation. Keep unknown facts unknown rather than hiding inference behind a canonical-looking value.

Comprehend detection versus Bedrock transformation: Comprehend returns operation-specific analysis fields and confidence scores. Bedrock can derive a domain-shaped record, but that flexibility requires a stronger semantic validation boundary.

Schema compliance versus evidence support: Structured outputs constrains supported output shapes. It does not establish that amounts reconcile, source identifiers exist, or a generated statement follows from the extracted original.

Architecture / decision scenario

A supplier-document pipeline retains extracted originals and creates normalized text views. It uses DetectDominantLanguage.Languages to choose a supported analysis path, then DetectEntities to enrich the view with organizations and dates. Each entity retains its type, score, offsets, and analyzed source version.

The pipeline uses Bedrock to turn those materials into a supplier review record. On a supported configuration, outputConfig.textFormat constrains the record’s basic shape. The application separately checks required identifiers against the source, reconciles monetary values, and enforces allowed ranges that the generation schema cannot express.

A record with valid JSON but an unsupported currency fails semantic validation. An entity with a low detection score remains available for review rather than becoming a verified supplier attribute. If provider citations are required, the team selects a compatible configuration instead of combining Anthropic citations with structured outputs.

Only records that pass the explicit validation result enter the consuming workflow. The original evidence and rejected derived record remain distinguishable, so correction does not rewrite the source.

Common exam traps

  • Treating normalization as permission to fill missing facts.
  • Applying offsets to a different text version from the one analyzed.
  • Treating Comprehend Score as factual truth confidence.
  • Assuming a custom recognizer obeys the request LanguageCode.
  • Assuming structured outputs supports every JSON Schema constraint.
  • Combining Anthropic citations with incompatible structured-output settings.
  • Accepting schema-valid records without checking source support.

What to remember

  • Retain originals and identify every transformed view.
  • Choose analysis operations by the enrichment the consumer needs.
  • Preserve Comprehend confidence, types, offsets, and language decisions.
  • Scope Bedrock structured outputs to supported models and schema features.
  • Validate completion, full schema, semantics, and evidence before release.

Knowledge check

References