NVIDIANVIDIA-Certified Professional: Agentic AI2026-101.5 Orchestrate multi-agent workflows and coordination.
Require measurable gains from specialist delegation
Completed by the knowledge check
Learning objective
Weigh multi-agent coordination overhead against single-agent simplicity.
Core idea
Multiple agents earn their cost when specialization, context isolation, parallel work, or independent team ownership solves a demonstrated problem. A complex task alone is insufficient justification. LangChain’s multi-agent documentation explicitly notes that a single agent with suitable, sometimes dynamic, tools and prompts can often achieve similar results.
Compare complete workflows, including delegation and synthesis. A specialist call can improve one subtask while making the total response slower or more expensive. Your architecture decision should state the quality gain and the latency, token, and operational costs needed to obtain it. NeMo Agent Toolkit evaluation and profiling provide concrete artifacts for that comparison.
How it works
Start with a competitive single-agent baseline
Give one agent the tools and context needed for the task, and evaluate where it fails. LangChain identifies excessive tool choice, extensive specialist context, parallelizable subtasks, and independent development as reasons to consider multi-agent patterns. Each is a hypothesis to test rather than a reason to add a supervisor automatically.
If the baseline selects the wrong tool from a large catalog, compare a specialist design that narrows each agent’s choices. If irrelevant context dominates the prompt, compare isolated specialist inputs. If the failure is an unavailable database or an incorrect source document, splitting the same task among agents does not address that cause. Keep the factual inputs and tool behavior comparable so the experiment tests coordination rather than unrelated changes.
A single agent can also load specialist context on demand through LangChain’s skills pattern. That preserves one control owner while reducing how much context must be present initially. It is a useful alternative when domain knowledge varies but the work does not need separate agent execution.
Account for every coordination call
In a supervisor design, include routing, specialist inference, tool calls, and final synthesis. Token accounting covers every model invocation, including repeated instructions and transferred results. A shorter specialist prompt does not guarantee a lower workflow total if three agents each repeat shared context and the supervisor rereads their outputs.
Separate total work from elapsed time. For three independent branches lasting two, three, and five seconds, ideal parallel execution takes at least five seconds before fan-in overhead, while sequential execution takes ten. Both execute all three branches. Add the supervisor’s routing and synthesis to the critical path; parallelism removes neither stage.
Shared inference capacity, tool contention, and queueing can move measured performance away from the ideal. Test with the intended request concurrency rather than assuming that isolated branch timings predict production latency.
Use NVIDIA’s profiler to expose the cost
Run candidate configurations through nat eval --config_file=EVAL_CONFIG_PATH, where EVAL_CONFIG_PATH is the evaluation YAML for the candidate. Evaluation executes the configured dataset and evaluates workflow outputs. The full evaluation runtime requires nvidia-nat[eval]; profiling also requires nvidia-nat-profiler.
The profiler documentation places settings under eval.general.profiler, with options including compute_llm_metrics. Framework instrumentation uses the framework_wrappers argument to register_function; confirm that custom specialist functions participate in usage collection before trusting totals. The profiler records per-invocation tokens, time between calls, and LLM calls, allowing you to distinguish specialist work from coordinator work.
Read workflow-runtime p95 from the nat eval console summary, which prints Workflow Runtime (p95) and LLM Latency (p95), and inspect workflow_profiling_metrics.json for aggregate means, percentiles, and summary statistics across the run. Keep standardized_data_all.csv for per-request usage inspection and all_requests_profiler_traces.json for per-request events. Use these profiler outputs to compare prompt and completion tokens, elapsed time, and failures across the same dataset. A missing span or uninstrumented external call is a measurement gap, not a free operation.
This lesson uses evaluation to decide whether coordination pays for itself. Metric design and tuning belong to section 3. Keep a release decision record containing the candidate configurations, workload, acceptance thresholds, and measured comparison; that record makes the architectural trade-off reviewable.
Bound the workload used for comparison
parallel_executor, see 1.5.2, runs tool_list branches concurrently; the overhead question is how this changes total work and concurrent demand.
Request-level concurrency multiplies branch demand. At ten concurrent requests and three simultaneously active branches per request, the workflow can have thirty active branches before considering nested calls. This is a workload calculation, not a GPU-sizing formula. Token lengths and model execution still determine the inference work.
During evaluation, eval.general.max_concurrency controls request concurrency and defaults to 8. NVIDIA documents reducing it, including setting it to 1, as a remedy for rate limiting during evaluation. A run that only succeeds after reducing concurrency is useful for correctness checks, but it does not establish latency at the intended production load. Rerun at the target workload after resolving the capacity constraint.
Key distinctions and trade-offs
Compare alternatives against the cause of the baseline’s failures:
| Design | Prefer when | Mechanism to test | Cost to measure |
|---|---|---|---|
| Single agent with tools | One context and control loop meet the task | Baseline workflow evaluated with nat eval | Tool-selection quality and total tokens |
| Single agent with on-demand knowledge | Domain instructions are needed selectively | LangChain skills pattern | Context-loading work and selection accuracy |
| Supervisor with specialists | Narrower contexts or tool sets improve results | Subagent invocation and final synthesis | Routing, duplicated context, and synthesis calls |
| Parallel specialists | Required analyses are independent | parallel_executor.tool_list | Critical-path time and concurrent branch demand |
Quality, latency, and tokens are separate axes. An architecture can improve quality and still violate the response budget. Parallel execution can improve elapsed time while spending more tokens. Independent ownership can justify a split even when inference savings are small, but the team must accept the additional integration and evaluation responsibilities.
Architecture / decision scenario
A publisher builds a research assistant for editorial briefs. Its product owner requires at least 90% accepted briefs, a six-second p95 response time, and no more than 8,000 model tokens per request on average. The following are hypothetical benchmark results over the same 500 briefs, with the same source collection and model configuration.
A single agent accepts 91% of briefs, uses 4,000 tokens on average, and finishes at 4.2 seconds p95. A supervisor with three specialists accepts 94%, uses 9,000 tokens, and takes 7.1 seconds. The tempting proposal is to deploy the specialist system because its quality score is higher.
The product owner rejects that release: the three-point quality gain breaches both the token and latency budgets. The architect keeps the single-agent workflow and records the specialist result as an optimization candidate. A skills-based variant is worth testing if the traces show that the single agent’s remaining errors come from missing domain instructions rather than inadequate source material.
If specialists perform independent analyses, test parallel_executor as another candidate. Do not assume concurrency fixes the token violation: all three branches still execute. Compare the complete route–analyze–synthesize path, then repeat with eval.general.max_concurrency set to the publisher’s target request concurrency rather than leaving the default 8. Compare p95 from the summary’s Workflow Runtime (p95) line and mean tokens from workflow_profiling_metrics.json, checking the per-request CSV for outliers. The platform owner reviews concurrent inference demand; the editorial owner reviews the accepted-brief failures. Deployment follows only if one candidate meets all three constraints.
Common exam traps
- Choosing multiple agents solely because the business task is complex.
- Comparing specialist latency while excluding routing and synthesis.
- Treating parallel execution as a reduction in total inference work.
- Counting only final-answer tokens and omitting coordinator and specialist prompts.
- Using a concurrency-one benchmark to claim a production latency target.
- Selecting a quality improvement that violates an explicit cost or latency constraint.
What to remember
Make multiple agents compete against a capable single-agent baseline. Measure the complete execution and identify which specialization benefit compensates for extra calls and coordination. Keep the simpler design when it meets the business thresholds and the multi-agent alternative spends more than the gain is worth.
Knowledge check
A legal-research assistant must have at least 88% of its answers accepted, stay under 5 seconds p95, and average at most 6,000 tokens per request. On the same 400-question set, a hypothetical benchmark gives a single agent 90% accepted, 3.8 seconds, and 3,500 tokens, and a supervisor with three specialists 93%, 6.4 seconds, and 8,200 tokens. Which release decision follows?
parallel_executorA supervisor routes each request in 0.5 seconds, runs three independent specialists that take 2, 3, and 4 seconds, and then synthesizes in 1 second. Assuming ideal parallel execution and no queueing, what changes when the team moves the specialists from sequential calls into a parallel_executor?
While evaluating a multi-agent candidate with nat eval, the team hits [429] Too Many Requests and sets eval.general.max_concurrency to 1. The rerun completes and reports a Workflow Runtime (p95) of 4.6 seconds against a 5-second target. Production is expected to carry 20 concurrent requests. What can the team conclude?
A single-agent support assistant fails 15% of evaluation questions. Its traces show that in nearly every failure it calls the right tool, but the product-documentation index returns an outdated manual. A proposal splits the agent into a supervisor with billing, product, and account specialists. What should the architect do first?