Choose chunks by the evidence needed

Learning objective

Choose fixed, structural, semantic, or parent-child chunking based on retrieval needs.

Core idea

For a retrieval architect, a chunk is the unit you can find, authorize, cite, and pass to the model. Choose that unit from the questions the system must answer. A paragraph can identify a relevant clause precisely, yet omit the exception that makes the clause usable. A whole document preserves context, yet represents several unrelated topics in one embedding.

Fixed, structural, and semantic chunking choose where text is split. Parent-child organization chooses the relationship between the smaller retrieval unit and the larger context you might return. You can combine these choices: structurally split a policy, retrieve a child passage, and then load its authorized parent section.

See 2.2.9 for embedding representation and 2.2.10 for query ranking. Chunking establishes what those mechanisms can retrieve; reranking cannot restore content discarded during ingestion.

How it works

Start with fixed windows when structure is weak

Azure AI Search provides Microsoft.Skills.Text.SplitSkill. With textSplitMode="pages", the skill creates length-bounded text chunks and attempts to avoid cutting sentences. Here, a page is a configured text window, not necessarily a physical PDF page. maximumPageLength bounds its length, and pageOverlapLength repeats the end of one chunk at the beginning of the next.

Character-based splitting is generally available. The default character maximumPageLength is 5,000, with a supported range of 300 to 50,000. Token-based splitting uses unit="azureOpenAITokens" and azureOpenAITokenizerParameters; the Text Split page marks these options preview. Do not describe a token value as though it were a character value.

Microsoft’s chunking guidance suggests starting experiments at 512 tokens with 128 tokens of overlap, or approximately 2,000 characters with 500 characters of overlap. These are starting points, not mandatory limits or guaranteed optima. Measure actual token counts for the embedding model and the assembled generation context.

Overlap helps when a question spans a split, but duplicates evidence and increases indexing volume. Deduplicate repeated passages before injecting context. The skill’s offsets, lengths, and ordinalPositions outputs help preserve each chunk’s position in its source.

Use structure when the document defines meaningful sections

Structural splitting follows headings, sections, paragraphs, or sentence punctuation. For sentence-based splitting, set textSplitMode="sentences". Microsoft maps content-characteristic grouping to the Azure Content Understanding skill or Text Split by sentences. Content Understanding’s default chunkingProperties.method="fixedSize" mode is generally available; its optional semantic mode is preview. For Markdown blobs, configure indexer parsingMode="markdown" and markdownParsingSubmode="oneToMany" to produce a search document for each content section. This is the documented one-to-many parsing path for heading-based organization. Keep a section’s title and source identity with its chunks so an isolated passage retains its subject.

Choose structure for manuals, contracts, and policy documents whose organization carries meaning. A troubleshooting procedure should retain its preconditions; a table should retain the labels that explain its cells. If a section exceeds model input limits, combine structural grouping with bounded subdivision rather than assuming every heading defines a safe-sized unit.

Structure is only as good as extraction. Inspect representative PDFs and tables after ingestion, including documents with repeated headers and page breaks. A boundary detector can be consistent while consistently separating an exception from the clause it modifies.

Select semantic grouping with its supported settings

Semantic chunking groups related content rather than applying only a character window. Azure AI Search’s Azure Content Understanding skill has a concrete semantic option: chunkingProperties.method="semantic" with unit="tokens". Its skill documentation marks semantic chunking preview, available starting with the 2026-05-01-preview API. The underlying skill is generally available in 2026-04-01; that status does not make this optional semantic mode generally available.

This semantic mode is layout-aware, respects paragraph boundaries, and handles large tables spanning chunk boundaries. Its maximumLength token range is 100 to 8,000. overlapLength must be omitted or set to 0 for semantic mode; it applies only to fixedSize mode. Do not transfer the fixed-window overlap settings unchanged.

Use semantic grouping when page or heading boundaries divide a coherent explanation, and evaluate whether the resulting units preserve the answer-bearing content. The feature’s semantic label does not prove superior retrieval for identifiers, short definitions, or every document family.

Separate child retrieval from parent expansion

Azure AI Search index projections support one-to-many indexing from a source document to chunk documents. A selector’s parentKeyFieldName connects the chunks to their parent. sourceContext identifies the enriched collection to project, and mappings carries the child content and inherited parent fields.

When chunks are the primary search target, parameters.projectionMode="skipIndexingParentDocuments" prevents extra parent-only documents from entering that chunk corpus. This controls indexing; it does not automatically implement a retrieve-child-and-expand-parent algorithm.

Microsoft recommends a single index with parent fields repeated on each chunk. Set the indexer and index-projection targetIndexName values to that same index and use projectionMode="skipIndexingParentDocuments". Separate parent and child indexes are an uncommon exception for application-specific requirements. Azure AI Search queries do not join those indexes: the application retrieves a child’s parent identifier, performs the parent lookup, and collates the result. Expand only as far as the question requires, and authorize the expanded content before it reaches another agent. See 2.2.11 for retrieval responsibility and authorization.

Index refresh also affects the relationship. Parent deletion removes projected children only when the data source’s dataDeletionDetectionPolicy detects the deletion; otherwise, unwanted children require manual deletion. A stale child can remain searchable after its source disappears if you omit this mechanism.

Key distinctions and trade-offs

The approaches address different needs:

Small chunks improve localization but can lose qualifications. Large chunks improve completeness but dilute representation and spend more generation context. Parent expansion offers a compromise only if you preserve source relationships and bound the expanded result.

Architecture / decision scenario

A support system ingests three document families. Short incident notes use fixed Text Split windows because their formatting is inconsistent. Product manuals use heading-based groups with the product and section title repeated as metadata. Complex policy PDFs are evaluated with the Content Understanding semantic preview mode to test whether paragraphs and tables remain coherent across page boundaries.

All three feed child documents with parent identifiers. A precise error-code query retrieves a small manual passage; the application loads its parent troubleshooting section only when the answer needs preconditions. A policy question retrieves a clause and includes the related exception from authorized parent context. Unnecessary whole-document expansion is rejected by the context budget in 2.2.3.

The team compares whether each approach finds the relevant passage and retains its necessary qualification. It configures deletion detection for supported source ingestion and verifies that withdrawn parents leave no searchable children. Chunk size, grouping, parent expansion, and refresh are evaluated together rather than treating one split setting as the complete retrieval design.

Common exam traps

• Confusing Text Split pages with physical document pages.
• Treating character counts as exact token counts.
• Presenting token-based Text Split or semantic Content Understanding options as generally available.
• Using nonzero overlap with Content Understanding semantic mode.
• Assuming index projections perform automatic query-time parent joins.
• Believing semantic reranking can recover a qualification omitted from the index.

What to remember

Choose splits from the evidence a question requires, and choose parent expansion separately. Preserve source positions and parent identifiers, verify feature-specific status, and evaluate answer completeness alongside retrieval precision. Refresh and deletion behavior are part of the chunking contract.

Knowledge check

References

• Microsoft, Chunk documents for vector search
• Microsoft, Text Split skill
• Microsoft, Azure Content Understanding skill
• Microsoft, Define index projections
• Microsoft, Index Markdown blobs and files