NVIDIANVIDIA-Certified Professional: Agentic AI2026-106.1 Implement retrieval pipelines (RAG, embedded search, hybrid approaches).
Size searchable chunks around usable context
Completed by the knowledge check
Learning objective
Choose chunking strategies: size, overlap, structure-aware, and parent-document.
Core idea
A chunk is both a search target and a possible part of the generator’s input. Those roles need different amounts of context. A short maintenance instruction can embed precisely yet omit the warning that makes it safe to apply. A complete manual preserves the warning but mixes so many subjects that its embedding becomes less useful for finding that instruction.
Choose the smallest searchable unit that preserves the meaning your questions need, then decide how much surrounding material to return. Size and overlap control boundaries mechanically; structure-aware splitting respects document organization; parent-document retrieval separates the search unit from the returned context. These choices combine rather than form four mutually exclusive algorithms.
How it works
Define the unit before tuning the number
LangChain’s RecursiveCharacterTextSplitter uses chunk_size, chunk_overlap, and length_function. With length_function=len, size is measured in characters. Its default separator order tries paragraph breaks, line breaks, spaces, and then individual characters. This preserves larger textual units while splitting oversized ones.
Do not copy a value from a character splitter into a token budget. The embedding model tokenizes the resulting text, and the generator has its own input budget. State the measurement alongside every proposed setting. A design record that says only “512” leaves the ingestion team and inference team free to implement different policies.
NVIDIA RAG Blueprint exposes APP_NVINGEST_CHUNKSIZE and APP_NVINGEST_CHUNKOVERLAP. Both values count tokens, unlike LangChain’s character counts with len. The settings page lists deployment defaults of 512 and 150; treat these as starting settings rather than a universal optimum. NeMo Retriever Library’s token splitter uses the revision-pinned nvidia/llama-nemotron-embed-vl-1b-v2 tokenizer, with max_tokens and overlap_tokens configured through .extract(split_config={"text": {...}}). Its plain-text and HTML default is 1,024 tokens; distinguish that library default from the Blueprint deployment settings. For paged files, APP_NVINGEST_ENABLE_PAGED_DOC_SPLIT=true enables chunk splitting after page-level extraction for PDF, DOCX, and PPTX. Leaving that switch false retains the page-level extraction behavior described by the Blueprint. Tuning chunk size without checking this switch can tune a stage that your paged documents never use.
Use overlap to repair boundary loss
Overlap repeats material between adjacent chunks so a sentence near a split remains connected to nearby context. LangChain describes chunk_overlap as a target, rather than a promise that every pair repeats an exact number of characters. Separators and the sizes of the available pieces affect the result.
Overlap increases duplicated indexed text and can cause several near-identical passages to occupy the retrieved candidate set. If a procedure crosses many boundaries, repeatedly increasing overlap can consume storage and answer context without restoring its overall structure. Inspect the split documents: a heading separated from its numbered steps is a different defect from a single sentence cut at the edge.
For a long, uniformly split text, an approximate chunk count is text length divided by chunk_size - chunk_overlap, ignoring edge effects. At a fixed size of 800 characters, increasing overlap from 80 to 240 changes the advance from 720 to 560 characters. That implies about 29% more chunks for the same long input.
Preserve structure, then split oversized sections
For Markdown, MarkdownHeaderTextSplitter(headers_to_split_on=...) groups text under selected headings and retains heading metadata. Set strip_headers=False when the actual heading text should remain in the returned passage. A title held only in metadata does not automatically become text the generator reads.
Follow header splitting with RecursiveCharacterTextSplitter.split_documents() when sections exceed your chosen size. This preserves section metadata and applies overlap within each section. It does not make overlap bridge unrelated header groups. Structure defines a meaningful boundary; the size cap still prevents a very long section from becoming one unfocused search target.
Structure-aware splitting depends on the structure you actually extracted. A scanned document with lost headings does not become structured because you choose a Markdown splitter. Inspect representative extracted pages before deciding which boundaries the splitter can honor. See 6.3 for ingestion and 6.4 for preprocessing; here the decision is what unit the retriever searches.
Search children and return parents
LangChain’s ParentDocumentRetriever searches small chunks, looks up their parent IDs, and returns the larger parent documents. Its child_splitter defines searchable pieces; parent_splitter can define larger returned sections. A parent can be a complete source document or a larger chunk of it.
This is useful when a small clause identifies the right material but answering requires its enclosing procedure or section. It also makes the size of that parent a serving decision. Returning a 200-page manual for every child hit replaces a retrieval problem with a context-budget problem. Prefer meaningful bounded sections when a complete source is too large.
Keep parent identity aligned with the version of the indexed child. If a child from one revision expands into a parent from another, the context can contradict the search hit. The ingestion owner should rebuild affected child-parent associations together when revising the source; the parent lookup is part of retrieval correctness, not a display convenience.
Key distinctions and trade-offs
Choose each control for the defect it repairs:
| Control | Deciding condition | Mechanism | Cost or failure risk |
|---|---|---|---|
| Size | Chunks mix unrelated topics or omit necessary context | chunk_size and length_function | Smaller units increase chunk count; larger units dilute focus |
| Overlap | Relevant meaning crosses an otherwise sensible boundary | chunk_overlap | Duplicate text consumes indexing and context capacity |
| Structure-aware | Headings or sections define meaningful units | MarkdownHeaderTextSplitter, then split_documents() | Requires usable extracted structure and a size cap |
| Parent-document | Search needs precision but the answer needs surrounding material | ParentDocumentRetriever, child_splitter, parent_splitter | Parent storage and larger returned context |
Chunking changes what later retrieval stages can find. A reranker cannot restore a warning absent from every candidate; see 6.1.3. Conversely, parent expansion can restore surrounding text without making every searchable embedding equally broad.
Architecture / decision scenario
A manufacturing assistant answers maintenance questions from versioned equipment manuals. Its application context allowance is 6,000 tokens. In an illustrative 120-question trial, arbitrary fixed chunks preserve the full instruction and warning in 78 cases; section-based parents preserve them in 109. Returning complete manuals requires over 20,000 tokens for several questions and violates the allowance.
The retrieval engineer chooses heading-based sections as parents and smaller recursively split children for search. They compare character sizes of 800 and 1,600 and target overlaps of 80 and 160, measuring the model-token lengths separately. The team rejects increasing overlap until whole procedures are duplicated: it raises chunk count while still failing on procedures longer than the overlap.
The content owner verifies heading extraction and parent revision identity. The serving owner limits how many section parents fit in 6,000 tokens. Together they choose the candidate that preserves warnings within that allowance, rather than the one with the most individually relevant child hits. The ingestion owner confirms APP_NVINGEST_ENABLE_PAGED_DOC_SPLIT before comparing chunk sizes on PDF manuals: with it false, page boundaries define the chunks.
Common exam traps
- Confusing LangChain’s character counts with
length_function=lenand NVIDIA’s token-based chunk settings. - Assuming paged documents always pass through additional chunk splitting.
- Using overlap to compensate for lost section structure indefinitely.
- Confusing a parent document with an obligation to return the complete original file.
- Judging child-search precision while ignoring the size and revision of returned parents.
What to remember
Size controls focus, overlap repairs local boundary loss, structure preserves meaningful sections, and parent retrieval separates search precision from answer context. Inspect the actual chunks and returned parents. The winning configuration fits the serving budget and preserves the complete material needed to answer.
Knowledge check
An ingestion team indexes PDF equipment manuals through the NVIDIA RAG Blueprint with APP_NVINGEST_ENABLE_PAGED_DOC_SPLIT set to false. They raise APP_NVINGEST_CHUNKSIZE from 512 to 1,024 tokens and re-ingest, but the passages retrieved from those PDFs do not change. Which change makes the chunk-size setting apply to these PDFs?
APP_NVINGEST_CHUNKOVERLAP to 512 before re-ingestingAPP_NVINGEST_ENABLE_PAGED_DOC_SPLIT to true before re-ingestingAPP_NVINGEST_CHUNKSIZE as characters before re-ingestingreranker_top_k to 20 before retrying retrievalAPP_NVINGEST_ENABLE_PAGED_DOC_SPLIT set to false , PDF, DOCX and PPTX files keep their page-level extraction, so page boundaries define the chunks and the chunk-size setting tunes a stage those files never pass through. Units, overlap and reranker retention are worth checking only once the splitter actually runs on these files.A maintenance assistant retrieves short instruction chunks precisely, but its answers often omit a safety warning stated elsewhere in the same procedure section. The warning chunks rarely match the wording of a question, and returning complete manuals would exceed the application’s 6,000-token context allowance. Some procedures are longer than any chunk the team would index, extracted headings are reliable, and the required procedure sections fit within the context allowance. Which design fits these constraints?
A team splits Markdown runbooks with MarkdownHeaderTextSplitter using its default settings, and every section already fits within its size cap. The pipeline passes only each chunk’s text to the generator, and answers confuse which equipment model a step belongs to, although the model name is the section heading. Which change addresses the problem while each returned passage stays one section long?
strip_headers=Falsechunk_overlap so adjacent sections share their headingschunk_size so each runbook fits into one chunkReferences
- NVIDIA, NeMo Retriever Library: Key concepts
- NVIDIA, RAG Blueprint ingestion configuration
- NVIDIA, Best Practices for Common NVIDIA RAG Blueprint Settings
- LangChain, Splitting recursively
- LangChain, How to split Markdown by headers
- LangChain, ParentDocumentRetriever