NVIDIANVIDIA-Certified Professional: Agentic AI2026-105.2 Apply reasoning frameworks (chain-of-thought, task decomposition).
Reserve tokens for a usable answer
Completed by the knowledge check
Learning objective
Apply chain-of-thought and reasoning-model modes within token budgets. For an agent architect, the decision is how much intermediate reasoning a request can afford while still producing a complete, usable answer.
Core idea
Chain-of-thought prompting asks a model to work through intermediate steps before answering. A reasoning-model mode controls how a particular model generates that intermediate work. Neither a longer prompt nor a longer reasoning trace proves correctness, and neither creates additional output capacity.
Separate three decisions: whether the task needs intermediate reasoning, which documented control selects that behavior, and how much generation remains for the answer. Model size, precision, and per-model reasoning controls belong to 3.4.2. Spend the selected model’s request budget across prompting, planning, execution, and the final answer. Use external checks for arithmetic, record identity, and tool results instead of spending more reasoning tokens to reconstruct authoritative data.
How it works
Distinguish prompting from model controls
An explicit step-by-step instruction changes the task presented to the model. Use it with an instruction-following model when naming intermediate decisions helps it choose tools or satisfy a multi-part request. Evaluate that prompt against a direct-answer baseline: the intermediate text earns its tokens only if the final result improves. For extraction or a deterministic transformation, a longer explanation can consume the answer allowance without establishing another fact.
NeMo Agent Toolkit’s react_agent makes prompt-level reasoning observable in its required output format: Thought:, Action:, and Action Input: precede a tool observation, and Final Answer: marks the result. This interleaves reasoning with tool feedback. The documented prompt requires {tools} and {tool_names} variables; use the available tool descriptions to ground its next-step reasoning. See 1.2.2 for parsing, retry, and termination behavior; the budgeting consequence is that each additional reasoning-and-tool cycle can introduce another model call.
When a model already has reasoning enabled, start with that documented mode and the required task or tool format. Add an explicit step-by-step instruction only if evaluation shows an incremental gain. Prompt-level intermediate text and built-in reasoning both spend generation capacity; requesting both does not itself establish a second quality benefit. A ReAct Thought: is part of the agent’s output protocol, while a model’s reasoning mode is an inference control. Preserve the protocol when testing a different mode.
NeMo Agent Toolkit also provides reasoning_agent as a workflow (_type: reasoning_agent) or a function. Its llm_name selects a configured LLM, and augmented_fn identifies the configured function that receives its plan. The default reasoning_prompt_template asks for a detailed step-by-step plan: identify the input’s key components, choose suitable tools, and outline the action sequence. This is an inspectable prompt-level planning mechanism, with another model call before execution.
The wrapper requires nvidia-nat[langchain] and an LLM that supports thinking tags such as <think></think>. Its documented flow reasons up front and does not revise the plan during execution. Test the selected model’s actual response representation against that requirement before using the wrapper.
Choose the reasoning approach within the budget
For NVIDIA-Nemotron-3.5-Super-VL-120B-A12B, reasoning shares max_tokens with the final answer; see 3.4.2 for reasoning_effort, enable_thinking, and the model’s budget-related failure and remedy. A per-call token ceiling can therefore terminate intermediate work before the answer is complete. Keep model controls separate from the application-wide allowance for all calls.
Choose the approach by the work it adds:
| Approach | Deciding condition | Mechanism | Aggregate-budget consequence |
|---|---|---|---|
| Direct answer | Extraction or a checked transformation is sufficient | Task prompt with the model’s documented direct-answer mode | Avoids an extra planning stage |
| Prompt-level intermediate steps | An instruction-following model needs explicit intermediate decisions or tool feedback | ReAct Thought: and tool-observation cycle | Reserve generation for every admitted cycle |
| Built-in reasoning | The selected reasoning model handles the task without an extra planning stage | Model-specific mode controls, as in 3.4.2 | Reserve both intermediate work and final-answer space |
| Plan-first wrapper | A separate execution plan improves downstream work | Toolkit reasoning_agent, reasoning_prompt_template, and augmented_fn | Reserve the planning call and subsequent execution calls |
Account for the whole request path
Maintain a request-budget worksheet with the input allowance, generation allowance for each planned call, admitted tool cycles, required final-answer space, and a stopping condition. This is an application artifact, not an extra NIM configuration key. If a wrapper first produces a plan and an executor then calls the model, account for both calls and for the plan becoming input to the executor.
For ReAct, max_tool_calls defaults to 15 and limits tool calls before answering. Set it deliberately for the request, but also count actual model calls, including parsing recovery, against the application allowance. A tool-call limit is neither a token ceiling nor a time deadline.
For example, a design allowance of 3,000 generated tokens for planning plus 5,000 for execution totals 8,000 before any correction pass. A correction call capped at 2,000 raises that ceiling to 10,000. These are illustrative application allocations, not NVIDIA defaults. A per-call max_tokens: 5000 does not constrain that whole sequence to 5,000 tokens.
Check both completion status and the expected answer field before advancing. An empty final answer after finish_reason: "length" is incomplete even if reasoning contains extensive text. Do not hand that text to a downstream tool as the requested result. Change the mode, simplify the task, or increase the allowed generation only when the request’s aggregate budget permits it.
Key distinctions and trade-offs
One long reasoning call and several short reasoning-and-tool cycles consume the budget differently. The second design can use fresh observations, but also repeats model input and adds tool latency. Measure the complete path before choosing it.
A reasoning trace is intermediate model output. A verified result is an answer checked against the task’s sources or acceptance criteria. Keep these separate in the result adapter: a fluent trace cannot replace a missing final answer.
Up-front planning can help a complex request but spends tokens before any tool supplies observations. Avoid the wrapper when a deterministic function can already complete the task. Its verbose setting also matters operationally: when enabled, the wrapper logs input, output, and intermediate steps. Choose that setting deliberately when requests contain sensitive business data.
Architecture / decision scenario
An insurance operations assistant summarizes a claim packet and recommends which missing documents to request. The service owner sets an illustrative ceiling of 8,000 generated tokens per request, a 1,000-token final-answer allowance, and a 6-second p95 target. End-to-end tests show a single full-reasoning design meeting the rubric on 97% of cases at 9 seconds. A single shorter-reasoning design reaches 96% at 5 seconds; direct extraction alone reaches 2 seconds but misses cross-document dependencies on 11% of cases.
The owner separately tests a composition: identifier extraction followed by shorter reasoning over the extracted records. Its measured end-to-end p95 is 5.5 seconds, with 98% rubric success: extracting identifiers first removes the cross-document identity misses behind most of the single shorter-reasoning design’s failures. That figure includes both stages and the final-answer check; it is not the sum of the single-call baselines. The owner chooses this composition, which buys two points of rubric success for one extra call and 0.5 seconds while staying inside the target, and caps extraction at 1,000 generated tokens and the reasoning call at 7,000, with answer space inside the latter allocation.
The tempting proposal is full reasoning plus a plan-first wrapper, but the full-reasoning baseline already misses the latency target. A proposed ReAct alternative uses max_tool_calls: 2 for targeted document lookups; its worksheet must also reserve every model turn and recovery call within the same 8,000-token ceiling and demonstrate end-to-end p95 below 6 seconds before replacing the measured composition. Cases that exhaust generation return incomplete for review. All measurements and allocations are scenario assumptions, not claims about a model’s performance.
Common exam traps
- Treating a step-by-step prompt, a reasoning mode, and a planning wrapper as the same control.
- Allocating
max_tokensentirely to the final answer even though reasoning shares it. - Assuming
"low"specifies an exact number of reasoning tokens. - Treating
max_tool_callsas a request-wide token or latency limit. - Counting only the executor call after introducing a reasoning wrapper.
- Accepting a long
reasoningfield whencontentis empty.
What to remember
Select intermediate reasoning for the task, then reserve generation for a complete answer. Use the selected model’s documented mode controls, account for every model call, and treat truncation as incomplete work. A planning wrapper adds a call and an output-format requirement; it does not provide a request-wide token budget.
Knowledge check
A claims assistant calls NVIDIA-Nemotron-3.5-Super-VL-120B-A12B through NIM with reasoning on and max_tokens: 1024. One response returns finish_reason: "length", an empty content, and a long reasoning field that already names the missing documents. What should the result adapter do with this response?
reasoning text downstream as the requested answermax_tool_calls so that the agent can finish the answermax_tokens with the answer, so a length stop can end inside the reasoning and leave content empty. That is incomplete work, and a fluent trace is not the requested result. Change the mode, simplify the task, or raise the allowance only if the aggregate budget allows; a step-by-step prompt spends more of the same allowance, and a tool-call limit is not a generation limit.A workflow runs NeMo Agent Toolkit’s reasoning_agent to produce a plan, then an executor model call, then an optional correction call. The design caps generation at 2,500 tokens for the plan, 4,000 for execution, and 1,500 for the correction, each set as that call’s own max_tokens. These are scenario allocations. How much generation must the request budget reserve?
max_tokens bounds one call, not the sequence. The wrapper adds a planning call before execution, and an admitted correction pass is another call, so the request ceiling is the sum of every call it may make. Counting only the executor, or leaving out the plan or the correction, understates what one request can generate.An agent built on react_agent must stay within 8,000 generated tokens per request. The engineer sets max_tool_calls: 2 and argues that this bounds generation. What does the design still need before it can be accepted against that ceiling?
max_tokens: 8000 on every model call in the loopmax_history: 2 setting to keep each turn’s input smallmax_tool_calls limits tool calls before answering; it is not a token ceiling. Each reasoning-and-tool cycle can add a model call, and parsing recovery adds more, so the worksheet must reserve generation for all of them within 8,000 tokens. A per-call cap of 8,000 lets the sequence exceed that, max_history counts messages, and a step-by-step instruction spends generation rather than bounding it.A ReAct agent runs on a model whose reasoning mode is enabled, and it meets its rubric. An engineer proposes also adding “think step by step” to the system prompt to raise quality further. What should the team do?
max_tokens to offset the extra reasoningThought: output formatmax_tokens shrinks the space the answer shares with reasoning, and the ReAct Thought: line is part of the agent’s output protocol, not a reasoning setting to swap out.