AWSAWS Certified Generative AI Developer - Professional2026-101.5 Design retrieval mechanisms for FM augmentation
Preserve evidence through parsing and transformations
Completed by the knowledge check
Learning objective
Configure Knowledge Bases parsing and Lambda custom chunk transformations.
Core idea
Parsing determines what information survives extraction; chunking determines how that information becomes searchable evidence. A transformation can reorganize parsed content or enrich its metadata, but it cannot reliably recreate an omitted figure or table from text that never represented it. Configure the ingestion stages as one evidence-preservation contract.
This lesson covers the configurable parser and Lambda transformation path for a self-managed Amazon Bedrock knowledge base. AWS also documents managed knowledge bases with different parsing strategies: SMART_PARSING, or MULTI_MODAL_EMBEDDINGS for native multimodal embedding models. Do not copy a configuration between those modes without checking the applicable API contract.
How it works
Choose the parser from required output
The Amazon Bedrock default parser extracts text from supported files, including PDF, Markdown, HTML, and Office document formats. AWS documents no usage charge for the default parser itself. That does not make embeddings, vector storage, or other ingestion operations free.
Use the default parser when text is the required evidence and extraction preserves its meaning. If the answer depends on figures, charts, tables, or images, AWS recommends Amazon Bedrock Data Automation or a foundation model parser. Those paths can extract multimodal elements and store extracted files in the S3 destination specified during knowledge-base creation. For a self-managed vector knowledge base, configure that destination through CreateKnowledgeBase.knowledgeBaseConfiguration.vectorKnowledgeBaseConfiguration.supplementalDataStorageConfiguration. Retrieval can return those files in its response or source attribution.
Bedrock Data Automation provides multimodal processing without an additional parsing prompt. Its parser cost depends on document pages or images. A foundation model parser supports a customized extraction prompt and charges based on input and output tokens. Select the latter when a tested extraction instruction changes the evidence you need, rather than assuming that prompt customization automatically improves fidelity.
For the foundation model parsing path, AWS documents a 100 GB total file-size limit across all files. This is a scoped parser constraint, not a general limit for every Knowledge Bases data source. Check it before allocating a large corpus to that ingestion path.
Configure the data-source interface
CreateDataSource carries parsing settings under vectorIngestionConfiguration.parsingConfiguration. Omit this configuration to use the default parser. For the advanced options, set parsingStrategy to BEDROCK_DATA_AUTOMATION or BEDROCK_FOUNDATION_MODEL and supply the corresponding configuration object.
Both bedrockDataAutomationConfiguration and bedrockFoundationModelConfiguration accept parsingModality: MULTIMODAL to enable parsing text and/or images; MULTIMODAL is the only valid value. The foundation model object is bedrockFoundationModelConfiguration. It includes modelArn and can include parsingPrompt.parsingPromptText. A custom prompt is therefore an ingestion instruction for extracting evidence, not the generation prompt used to answer the final question. Verify that the chosen parsing model and Region support this use before treating an inference-capable model as an eligible parser.
An advanced parser selection applies to all PDF files in that data source, including PDFs containing only text. The normal parser does not selectively replace it for inexpensive text-only PDFs. Separate data sources when your desired parsing policy differs between text documents and visual reports.
There is an important exception: the ParsingConfiguration API documentation says the default parser is used if the selected Bedrock Data Automation or foundation model parser fails to parse a file. Distinguish this failure fallback from normal PDF processing. A successful ingestion can therefore require output inspection to establish whether the required visual evidence survived; success alone is not an extraction-quality guarantee.
Add custom chunks or chunk metadata
Configure vectorIngestionConfiguration.customTransformationConfiguration with intermediateStorage.s3Location.uri and a transformations entry. That entry contains transformationFunction.transformationLambdaConfiguration.lambdaArn and stepToApply: POST_CHUNKING.
For custom chunk boundaries, combine this transformation with chunkingConfiguration.chunkingStrategy: NONE. Knowledge Bases writes content to the intermediate S3 bucket; your function reads it, applies the custom chunking logic, writes transformed files back to that bucket, and returns references for subsequent processing. NONE prevents a built-in strategy from dividing the document before your logic owns the boundaries.
For metadata enrichment, retain a native chunking strategy and run the same post-chunking transformation. Your function receives already parsed and chunked content, so it can attach chunk-specific attributes such as section identity or an effective date. Chunk-level metadata takes precedence over file-level metadata when keys collide. Treat that overwrite behavior as part of the contract, especially for fields that influence retrieval filters.
The function does not receive an arbitrary document body directly in the invocation event. The documented input includes version, knowledgeBaseId, dataSourceId, ingestionJobId, bucketName, and inputFiles. Each input file includes originalFileLocation, fileMetadata, and contentBatches containing S3 object key values.
The referenced content objects use fileContents, whose entries contain contentBody, contentType, and contentMetadata. The Lambda response returns outputFiles with source locations, file metadata, and output contentBatches keys. Preserve the original source association while returning keys for actual written objects. Returning an invented key or losing the original location breaks the intended traceability even if the transformation code produces plausible text.
Content moves through the intermediate bucket, and Knowledge Bases and the function exchange S3 object keys, not document bodies.
For web connectors, AWS states that Lambda receives Markdown rather than HTML. Write the transformation for the actual intermediate representation. An HTML-specific selector is not a general solution for every connector.
Key distinctions and trade-offs
Keep three responsibilities separate:
| Responsibility | Mechanism | Evidence to inspect |
|---|---|---|
| Extract source content | parsingConfiguration | Whether text, table values, and visual relationships survive |
| Define retrieval units | Native chunking or NONE plus Lambda | Whether each output chunk contains a coherent evidence unit |
| Enrich searchable attributes | POST_CHUNKING transformation | Whether contentMetadata preserves the intended filter meaning |
Custom transformation adds control and an application-owned file contract. Native chunking avoids that implementation burden when it already preserves the document’s boundaries; see 1.5.1. Neither approach makes parser output trustworthy without checking representative source-to-output samples.
An intermediate S3 bucket is a working artifact store. AWS supports an optional KMS key for its stored files. Account for bucket access and encryption permissions in the deployment design, and test the function against the references it actually receives rather than against a manually pasted text string.
Architecture / decision scenario
A manufacturing assistant searches text-only procedures and inspection PDFs containing annotated diagrams. It also needs each retrieved requirement tagged with its equipment family and section identifier.
The team uses separate data sources. Text-only procedures retain the default parser. Inspection PDFs use Bedrock Data Automation because visual evidence must survive without a custom extraction prompt. A subset with a specialized table interpretation is evaluated with a foundation model parser and an explicit parsingPromptText.
For procedures, native chunks receive section metadata through a POST_CHUNKING Lambda. For reports whose custom boundaries follow inspection records, the team selects NONE and lets Lambda produce record-level chunks in the intermediate bucket. It compares output objects with the originals and checks metadata collisions before releasing ingestion. If the advanced parser falls back to text extraction, the team withholds documents whose answers depend on omitted diagrams rather than treating an ingestion completion as proof of completeness.
Common exam traps
- Using a custom chunker to fix information lost by the parser.
- Assuming an advanced parser charges only for visually complex PDFs in its data source.
- Configuring a custom transformation at an undocumented pre-parsing stage.
- Returning chunk bodies where the contract expects S3 batch references.
- Assuming file-level metadata always overrides chunk-level values.
- Treating managed and self-managed parsing enums as interchangeable.
What to remember
- Select the parser for the evidence that must survive extraction.
- Advanced parsing applies across the data source’s PDFs, with a documented failure fallback.
- Use
NONEplusPOST_CHUNKINGfor custom boundaries. - Preserve S3 object references, source locations, and metadata semantics.
- Validate extraction output separately from ingestion completion.
Knowledge check
Inspection PDFs carry annotated diagrams that answers depend on. The team has no specialized extraction instruction to supply and prefers parsing cost to follow the number of pages processed. Which data-source configuration fits?
POST_CHUNKING Lambda that describes each diagramBEDROCK_FOUNDATION_MODEL with a custom parsingPromptTextBEDROCK_DATA_AUTOMATION with parsingModality set to MULTIMODALSEMANTIC chunking around each diagramA Lambda function, not a built-in strategy, must decide the chunk boundaries for a self-managed knowledge base. Which configuration does that?
FIXED_SIZE chunking with a POST_CHUNKING transformationNONE chunking with a POST_CHUNKING transformationNONE chunking with a pre-parsing transformationSEMANTIC chunking with a POST_CHUNKING transformationNONE keeps each document whole, so the post-chunking function receives undivided content and owns the boundaries. With a native strategy the same transformation receives chunks that are already split, which suits metadata enrichment rather than custom boundaries, and POST_CHUNKING is the only documented step.A transformation Lambda produces correct chunks and returns their text inline in its response. Why does this break the documented contract?
outputFiles whose contentBatches hold the keys of those objects alongside the original file location. Correct text returned any other way leaves ingestion with nothing to process and loses the source association.File-level metadata sets effective_date from a document’s cover page, and a POST_CHUNKING function also writes effective_date on each chunk from its section heading. Retrieval filters on effective_date. Which value does a chunk carry?
An advanced parser fails on a visual report, and the default parser extracts its text instead. Ingestion completes, but answers need values from a chart in that report. Which release decision preserves evidence quality?