NVIDIANVIDIA-Certified Professional: Agentic AI2026-101.6 Apply logic trees, prompt chains, and stateful orchestration for multi-step reasoning.
Choose rules before delegating semantic routing
Completed by the knowledge check
Learning objective
Implement routers and decision trees (LLM-based versus rule-based)
Core idea
A router chooses a destination. A decision tree composes choices so that each result determines which question or operation follows. Choose rules when trusted fields determine the answer; choose an LLM when understanding free-form intent supplies information that those fields lack.
The important design decision is who owns each predicate. An LLM can classify a request’s topic, while code enforces the destinations permitted for the authenticated request. A routing decision is neither authorization nor proof that the destination can complete the task.
How it works
Put explicit predicates in code
A rule-based router evaluates known inputs: a request type, an account’s region, or a validated status field. In Python LangGraph, StateGraph.add_conditional_edges() connects a routing function to named destinations. That function can inspect state and return a label that an explicit mapping resolves to a node. It need not call a model.
For example, an employee request marked as a payroll inquiry can follow a payroll branch after a code predicate checks the trusted request category. Define what happens when a field is absent or a label is unsupported. An application-defined clarification node is a real destination, not an invitation for the model to invent another handler.
Order matters in a decision tree. Evaluate restrictive predicates before semantic classification when they determine which handlers are permitted. A tree that first chooses a specialist and then discovers that the specialist cannot process the request wastes work and risks sending data to the wrong processor. Keep the authority-bearing predicate tied to authenticated application data rather than a user’s claim inside the request text.
Rule-based routing costs code maintenance and requires cases you can enumerate. It is attractive when a small set of stable predicates covers the workload. It becomes brittle when the rules attempt to enumerate every paraphrase of an open-ended question. That is where semantic classification can justify its inference cost.
Configure a semantic router
NeMo’s router_agent, introduced in 1.2.1, selects one branch per request.
Make descriptions distinguish the destinations by the decision the router must make. Two branches both described as “answers employee questions” provide little useful separation. “Explains payslip deductions” versus “Explains leave balances” gives the classifier a task-specific distinction. The service owners must agree which destination handles an ambiguous question before prompt wording can express it.
system_prompt can override the default routing instructions. A custom user_prompt template must include variables for input and available branches. Changing these prompts changes the classifier’s decision conditions; it does not change the permissions of the selected function. Keep access enforcement in that function or its surrounding application logic.
Distinguish invalid selection from wrong selection
max_router_retries defaults to 3 and limits retries when the agent fails to choose a valid branch. A branch name outside the configured choices is invalid. A permitted payroll branch selected for a leave question is valid but incorrect. Raising the retry setting does not address that second problem.
For invalid selection, inspect the routing instructions and configured choices. For valid-but-wrong selection, inspect overlapping descriptions and representative requests, including requests containing multiple intents. NeMo’s detailed_logs: true exposes routing decisions and branch responses for diagnosis. Its log_response_max_chars defaults to 1,000 and controls displayed branch-response length; choose logging content appropriate to the application’s data rules.
Retries answer only an invalid selection; a valid-but-wrong selection calls for inspecting overlapping descriptions instead.
LangChain’s documented routing pattern uses llm.with_structured_output(Route) with a Pydantic result whose step is a Literal of allowed labels. A conditional-edge function then maps that decision to a graph node. This separates a schema-constrained classification from execution. As in 1.6.1, a valid shape does not establish semantic correctness: the returned label can belong to the enum and still describe the wrong intent.
Compose the tree without implying a planner
NeMo documents both router_agent and sequential_executor as usable under functions; a router’s branches can name those functions. A selected branch can therefore be a configured pipeline or another router. This creates a decision tree whose internal choices occur at explicit boundaries.
A nested tree is still different from an agent that repeatedly chooses actions after observations. Two router levels mean two explicit classification points; they do not turn the top-level router into a multi-branch planner. Use 1.6.3 when the workflow needs recurring decisions over shared execution state.
Each semantic level adds inference work before the specialist begins. Put a rule-based split ahead of an LLM split when the rule resolves a large fraction of requests from trusted fields. Do not add an LLM at every branch merely to make the diagram look uniformly agentic.
Key distinctions and trade-offs
Choose the predicate by the information it needs:
| Option | Deciding condition | Mechanism | Cost | Owning lesson |
|---|---|---|---|---|
| Rule-based route | Trusted fields determine destination | Conditional-edge function and explicit label mapping | Code changes as cases evolve | 1.6.2 |
| LLM-based route | Meaning of free-form text determines destination | router_agent, branches, llm_name | Inference latency, tokens, classification errors | 1.6.2 |
| Composed decision tree | Several bounded questions select a final handler | Nested routers or graph conditional edges | More levels increase latency and maintenance | 1.6.2 |
| Adaptive action loop | Observations determine repeated next actions | State graph and bounded feedback edges | Variable execution length | 1.6.3 |
Rules make decisions reproducible for the same validated inputs. Semantic routing tolerates linguistic variation but requires evaluation of classification quality. Combining them lets each solve the part it can determine; it does not remove either owner’s responsibility.
Architecture / decision scenario
A university’s employee assistant handles 10,000 requests per day. Its authenticated portal supplies an explicit category for 70% of requests; 30% arrive as uncategorized text. The application budget permits 1.5 seconds for routing before the specialist starts. In the team’s trial, a conditional-edge rule takes 5 milliseconds, while semantic classification takes 600 milliseconds at the measured percentile.
The platform owner considers three proposals: classify every request with an LLM, maintain keyword rules for every text request, or use trusted category rules followed by semantic classification only for uncategorized requests. The classifier’s daily allocation permits 4,000 requests, ruling out classifying all 10,000. On uncategorized text, keyword rules misroute 12% of trial requests, versus 3% for semantic classification; the service owner’s acceptance threshold is 5%. The hybrid design sends 3,000 daily requests to the classifier and meets both constraints.
The owner chooses category rules, then a single router_agent among permitted specialist functions. The payroll and leave service owners define distinct descriptions and an application-defined clarification branch for unresolved intent. A trial’s wrong-but-valid selections trigger description and evaluation changes; malformed selections exercise max_router_retries. Authorization checks remain in the specialist functions. A question requesting both payroll and leave work is clarified or handled by an explicitly composed workflow, rather than assuming this single-branch router executes both.
Common exam traps
- Treating an LLM’s selected branch as permission to access its data.
- Increasing invalid-selection retries to fix valid-but-wrong classifications.
- Treating a
Literal-constrainedRouteas proof that the classification is correct. - Using prompt text instead of trusted state for a restrictive predicate.
- Calling a nested decision tree an unrestricted adaptive planner.
Where the exam guide differs
The study guide says “logic trees”. NeMo Agent Toolkit documents a single-pass “router agent” and compositions of routers and executors. Read an exam item’s tree as explicit branch decisions; do not infer that router_agent performs recursive planning, branch combination, or inter-branch communication.
What to remember
Use rules for explicit predicates and an LLM for semantic intent. Name every permitted destination, handle unresolved cases, and enforce access independently of classification. NeMo’s router selects one branch; nested routers add explicit decisions and additional inference cost.
Knowledge check
A NeMo Agent Toolkit router_agent chooses between payroll_help and leave_help, and both branch descriptions read “Answers employee questions”. Evaluation shows that many leave questions go to payroll_help, while logs show every selected name is one of the configured branches. Which change addresses the misrouting?
max_router_retries from its default of 3detailed_logs: true on the routerLiteral of the two branch namesmax_router_retries only retries when the router fails to choose a valid branch. Identical descriptions give the classifier no task-specific distinction to decide on; a Literal constraint enforces a shape the selections already have, and detailed logs help diagnose without changing any decision.An HR assistant uses an LLM router to send salary questions to a compensation_details function that reads individual pay records. Only managers may see other employees’ pay. Where should the team enforce that rule?
compensation_details , using the authenticated user’s rolesystem_prompt , as an instruction about managersAn insurer’s internal helpdesk receives 20,000 requests a day: the ticketing system sets a trusted ticket type on 12,000 of them, and 8,000 are free text. The LLM classifier’s daily allocation is 10,000 requests. On free-text trial requests, keyword rules misroute 15% and the classifier 4%, against an acceptance threshold of 6%. Which routing design meets both constraints?
An assistant’s NeMo Agent Toolkit router_agent has payroll_help and leave_help branches. Some messages ask to change a tax withholding and book leave at the same time, and users report that only one part gets answered. What should the design do with such requests?
max_router_retries for the routersystem_prompt to select both branchesrouter_agent under each of the two branchesReferences
- NVIDIA, Configure the Router Agent
- NVIDIA, Configure With the Sequential Executor
- LangChain, Workflows and agents
- LangChain, Graph API overview
- NVIDIA, NVIDIA-Certified Professional: Agentic AI Exam Study Guide