Assemble context from validated request state

Learning objective

Compose dynamic prompts from validated state, retrieved context, and tool results.

Core idea

A dynamic prompt is a request assembled for a particular execution step. Its content depends on the state you have accepted, the source passages selected for the question, and the observations returned by tools. Reliability depends on that assembly as much as on the instruction text.

See 1.6.1 for validating intermediate chain results and 2.1.1 for system instructions. Here the decision is which accepted information reaches this model call, in which message role, and with enough identity to interpret it. Copying every available string into a growing system prompt destroys those distinctions and spends tokens on information the step may not need.

How it works

Render a deliberate view of state

Use an explicit input model for the prompt-assembly function. NVIDIA NeMo Agent Toolkit functions support Pydantic input and output schemas, and FunctionInfo.from_fn() wraps a callable as a function definition. A custom registered function can turn typed request data into the message sequence used by its model call. Keep this assembly in one callable so its decisions can be tested without a live generation.

For example, an application-defined request model might contain a validated question, permitted document identifiers, locale, and the execution step. Render the fields needed for that step. A locale can control response language; it does not establish which documents the caller may read.

Keep authenticated access decisions outside model-authored text. A generated summary that says “administrator” is not the source of the caller’s privileges. OWASP recommends least privilege and separating external content. Apply those controls before a passage enters the prompt, because a later instruction cannot undo disclosure to the model endpoint.

Define the difference between missing, empty, and failed data in the assembly input. An empty list of matching passages can be a valid retrieval result. A failed retrieval is an operation outcome. Rendering both as an empty context block encourages the generator to treat an outage as absence of relevant information.

Select retrieved content as observations

Retain the connection between a passage and its source identifier. Give the model the passage text it needs and the identifiers it must cite. In LangChain, a retrieval tool can return a ToolMessage whose content reaches the model and whose artifact holds supplementary application data that does not.

Use that separation deliberately. If a citation identifier exists only in artifact, the model has not received it. Put the citation label beside the passage in content and keep the full document record or rendering metadata in artifact. The user interface can resolve the label against the retained record rather than trusting an invented URL in generated text.

A retrieval result is source material, not an instruction update. OWASP’s prompt-injection guidance describes indirect injection through external documents and recommends clearly identifying untrusted content. Keep retrieved text distinct from SystemMessage instructions and label its origin. Delimiters help represent the distinction, but they do not establish that the model cannot follow an instruction hidden inside the passage.

Resolve insufficient or conflicting sources explicitly. If the selected passages cannot support the requested conclusion, render that state and instruct the answer step to identify the gap. Do not fill the missing portion with unrelated high-similarity passages merely to make the context look complete. Retrieval quality itself belongs to section 6; assembly preserves what the selected sources actually say.

Associate tool observations with their calls

LangChain represents a model’s tool request in an AIMessage with tool_calls. A returned ToolMessage.tool_call_id must match the ID of the call it answers. Preserve the requesting message and the result together. Tool output that is attached to the wrong request can be structurally readable while describing a different entity or time.

For an application-defined retrieval call, construct the result message from the selected passage and its source record:

from langchain.messages import ToolMessage

result_message = ToolMessage(
    content="[policy-17] Exchanges require the original receipt.",
    tool_call_id="call_123",
    name="lookup_policy",
    artifact={"document_id": "policy-17", "revision": "r4"},
)

This fragment assumes the preceding AIMessage requested lookup_policy with ID call_123. Only content is the model-visible observation; artifact remains available to application code.

Retain a result’s success or failure meaning when you summarize it. If a stock lookup fails, do not render “out of stock.” Include only the permitted, relevant observation in the model-facing message, while retaining diagnostic details in the application’s result record. A tool output can carry hostile text too; a successful transport does not make its prose trusted instructions.

Budget the assembled request

Choose a per-step input allowance and reserve room for the expected answer. Token-cost concepts are covered in 0.2; the assembly-specific choice is what to omit. Drop irrelevant fields and duplicate passages before removing source identity or the tool-call association needed to interpret a result.

For NIM LLM deployments using the vLLM-backed API, POST /v1/chat/completions/render returns the formatted prompt without inference. Render the assembled messages to inspect role placement and tool results in the model’s chat template, then use POST /tokenize to count the rendered input against the per-step allowance; see 0.2. Check the container’s /docs for the accepted request fields.

Keep the stable instructions distinct from changing observations. Replacing policy prose with a larger transcript to avoid writing an assembly function gives every future step an expanding input. ReAct’s max_history bounds conversation history by message count and defaults to 15; it does not replace the token allowance. A typed assembly function makes the inclusion rule explicit: which state fields, which passages, and which tool outcomes belong to this question.

Record the template revision and the selected source identifiers with the execution record. That lets you distinguish a changed template from changed context when a regression occurs. It is the input to the version comparison in 2.1.4, not proof that the generated answer is correct.

Key distinctions and trade-offs

The three dynamic inputs serve different roles:

More context can improve coverage while increasing input cost and exposing more distracting text. A smaller prompt is useful only if it preserves the information needed to decide and cite the answer.

Architecture / decision scenario

A retail returns assistant must respond within a three-second service target. Its application budget allows 4,000 input tokens for the answer step. A full customer transcript plus every retrieved policy passage uses 16,000 tokens; a selected view uses 2,800.

A practitioner proposes increasing the input allowance and concatenating everything into SystemMessage. The workflow owner instead validates order state, selects the permitted returns-policy revision, and keeps the stock result associated with its original tool call. The assembly function puts the relevant passages and citation labels in content, retains document records in artifact, and renders a failed stock lookup as unavailable information.

The team chooses the selected view because the larger alternative exceeds the application budget and promotes customer and document text into instructions. The prompt owner maintains inclusion rules, the retrieval owner supplies scoped sources, and the tool owner preserves result identity. An answer that promises replacement stock after a failed lookup fails acceptance even if its policy citation is correct.

Common exam traps

• Treating validated state as permission to expose every field to the model.
• Moving retrieved instructions into the system message because retrieval succeeded.
• Expecting the model to cite an identifier stored only in artifact.
• Matching tool output by arrival order instead of tool_call_id.
• Rendering a failed lookup as a successful result with an empty value.

What to remember

Compose a deliberate view for each step. Validate state, preserve source identity, associate each tool observation with its call, and keep external material distinct from instructions. Reduce context by relevance rather than by deleting the identity needed to interpret it.

Knowledge check

References

• NVIDIA, API Reference
• NVIDIA, Configure the ReAct Agent
• NVIDIA, Functions
• NVIDIA, Writing Custom Functions
• LangChain, Messages
• OWASP, LLM01 Prompt Injection