NVIDIANVIDIA-Certified Professional: Agentic AI2026-105.4 Manage stateful orchestration to coordinate complex tasks and knowledge retention.
Isolate sessions and reconcile concurrent updates
Completed by the knowledge check
Learning objective
Isolate per-session and per-user state under concurrent updates.
Core idea
Isolation asks whose state a request can access. Concurrency control asks what happens when two authorized requests change that state together. Different users need separate state, but two sessions belonging to the same user can still overwrite each other’s work. A per-user component solves a different problem from an atomic update.
See 1.4.2 for authenticated scopes and memory lifecycle. Here the design adds simultaneous requests, multiple conversations, and serving replicas. Keep a session’s execution cursor separate from reusable user knowledge, then choose a synchronization mechanism at the scope where conflicting writes actually occur.
How it works
Identify each scope independently
A LangGraph checkpointer uses configurable.thread_id to associate state with a thread. A store uses namespace tuples and item keys for knowledge shared across threads. Assign separate thread IDs to independent tasks, even when one person starts them both. Reusing one user ID as every conversation’s thread ID merges execution that should remain independent.
For retained knowledge, an application-defined namespace such as (tenant_id, user_id, "preferences") distinguishes the user’s shared facts from a single task. Include both tenant and user when the application’s identity system requires them. LangGraph store namespaces can represent arbitrary scopes; they do not themselves establish whether the requester is authorized to use a chosen namespace.
Keep lookup construction on the trusted application path. LangGraph store.search takes a namespace prefix: a shorter prefix can include multiple child namespaces. Use the full intended scope and inspect returned namespaces when exact-level results matter. A broad tenant prefix is a different query from one user’s preference namespace.
Use NeMo Agent Toolkit per-user instances deliberately
Use @register_per_user_function for isolated runtime instances, recalling the construction, schema, and dependency rules in 1.8.3. @register_per_user_function_group shares state among functions for that user; inspect shared-function dependencies so user-specific mutable state stays inside its intended scope.
The FastAPI front end derives per-user workflow routing from the nat-session cookie, with a decoded JWT fallback. NVIDIA explicitly warns that this JWT payload is not verified and must not establish authorization without verification elsewhere. Memory identity follows the separate SessionManager.session(user_id=...) path in 1.4.2. Conversation-aware memory backends can additionally use conversation_id.
Normalize those identities through verified application authentication. A cookie used to select an instance and a user ID used to select retained memories are not automatically the same business scope. Two browser sessions can require distinct running workflows while still accessing the same authenticated user’s preferences.
Protect read-modify-write operations
Suppose two requests read the same preference record, revise different fields, and each replace the entire record. Per-user routing keeps other users out, but both requests remain legitimate writers to the same user’s record. The second write can erase the first update.
Within one Python event loop, asyncio.Lock provides exclusive access for cooperating tasks. Acquire a lock keyed to the conflicting state, reread the value inside the protected section, validate the change, and write it before releasing the lock. Use async with lock for release on exit. Holding one global lock across all users would serialize unrelated work; holding a lock across model inference also makes that user’s other updates wait for inference latency.
Python documents that asyncio synchronization primitives are not thread-safe. Use this mechanism for cooperating tasks in the same event loop. Two serving processes each constructing their own lock do not share that object, so a replicated deployment needs coordination in the shared persistence path.
For a Redis-backed application record, use WATCH before reading, then queue the revised write with MULTI and commit with EXEC. If a watched key changes before execution, EXEC aborts with a null reply. Reread and recompute instead of retrying the stale replacement. This is an application record adapter using Redis commands, not a NeMo memory configuration flag.
When two requests watch and read the same record, the first EXEC commits, and the second aborts with a null reply, so that request rereads and recomputes.
Keep this transaction short. Redis serializes the queued commands, but does not roll back commands that fail during execution. Inspect execution results; do not treat the transaction as protection for an external API effect. Under contention, bound application retry attempts and return an explicit conflict when the budget expires. The write owner decides whether competing edits merge or require user resolution.
Separate instance cleanup from durable state
Recall per_user_workflow_timeout and per_user_workflow_cleanup_interval from 1.8.3: they control inactive instances, not durable knowledge retention or checkpoint recovery. Set them for the resource cost and expected interaction gaps of your service.
Persist required knowledge and execution progress through their respective backends before relying on instance recreation. A timeout-based cleanup can release a user’s connections and local objects while their durable task remains resumable. Conversely, retaining a local instance longer does not demonstrate that another replica can recover its state. See 5.4.2 for durable checkpoint recovery.
Key distinctions and trade-offs
Match the mechanism to the competing writers:
| Scope | Mechanism | Deciding condition | Cost or limitation |
|---|---|---|---|
| Independent task sessions | Separate configurable.thread_id values | Tasks must progress independently | More checkpoint histories; shared user facts still need a store |
| User-local component state | @register_per_user_function | Users need separate runtime instances | Per-user resources and cleanup; same-user races still need coordination |
| Cooperating tasks in one event loop | Per-record asyncio.Lock | All conflicting writers share the lock object | Contention waits; no coordination across separate processes |
| Shared Redis record | WATCH, MULTI, EXEC | Replicas perform conditional read-modify-write updates | Conflicts require rereading; runtime errors do not trigger rollback |
Graph-state ownership and user-knowledge ownership can differ. Keep one active writer for a task’s control decisions in your application integration; independent task writers can still contend on a shared preference record. A namespace chooses the record, while locking or conditional execution protects the update.
Architecture / decision scenario
A construction firm’s meeting companion supports 80 project managers. Each can run three simultaneous meeting tasks across two serving replicas. Each meeting has its own action list, but all three may update the manager’s preferred summary format. Requests must not expose another manager’s notes.
A developer proposes one per-user workflow with one thread_id equal to the manager ID and an in-process lock. The platform lead rejects that combination: it merges three independent action lists into one execution thread, and two replicas have separate lock objects.
You assign a distinct configurable.thread_id per meeting. Your trusted application passes the verified manager ID through SessionManager.session(user_id=...) for memory identity and combines the verified tenant and manager IDs in the preference namespace. For runtime routing, it supplies a distinct nat-session cookie value per meeting session, bound to that authorized meeting by the application. The toolkit therefore treats the three cookie values as three per-user instance identities, while retained preferences remain manager-scoped.
Across the two replicas, treat these instances as replica-local resources rather than the owner of the shared preference record. A meeting session reaching another replica must recover its durable task state; do not depend on the first replica’s local objects. Protect the shared preference update with the Redis conditional transaction. If two sessions read revision 7, the first commits revision 8; the second detects the change and recomputes against revision 8 rather than restoring a stale copy. Revision fields are application-defined.
The workflow owner tests concurrent task progression and same-record conflicts. The identity owner tests forged routing values, and the platform owner checks instance cleanup and recovery across replicas. The design preserves three task histories while reconciling one shared user record; increasing the timeout or adding more replicas cannot substitute for either distinction.
Common exam traps
- Equating user isolation with serialized updates from the same user.
- Giving every task for one user the same checkpoint thread ID.
- Treating decoded JWT claims or a client routing cookie as verified authorization.
- Assuming per-user workflow routing and memory identity use interchangeable inputs.
- Expecting a local asyncio lock to coordinate independent serving processes.
- Retrying a stale replacement after a conditional-write conflict.
- Treating instance cleanup timeouts as durable knowledge retention settings.
What to remember
Scope the execution thread to the task and retained knowledge to its authorized sharing domain. Per-user instances prevent accidental component sharing; they do not settle simultaneous updates. Use local locks only where writers share them, and conditional shared-store writes where replicas compete.
Knowledge check
A meeting companion lets each project manager run three meetings at once. Each meeting has its own action list. The workflow uses the manager’s ID as configurable.thread_id, and action items from one meeting appear in another. Which change fixes the merging?
@register_per_user_functionconfigurable.thread_id to each meeting taskper_user_workflow_timeout for the idle meeting instancesTwo serving replicas update a manager’s shared preference record by read-modify-write. Each process guards the update with a per-record asyncio.Lock, and requests for one manager can reach either replica. Updates are still lost. The record lives in Redis. Which change prevents lost updates?
asyncio.Lock@register_per_user_functionWATCH , MULTI , EXEC conditional writeTwo requests watch and read revision 7 of a preference record, each revising a different field. The first commits revision 8. The second request’s EXEC returns a null reply. What should the second request do?
A NeMo Agent Toolkit FastAPI service routes requests to per-user workflow instances. Clients without a nat-session cookie send an Authorization: Bearer token, and any client can send a token or cookie value it constructed itself. The application must never return one manager’s notes to another. How should the application choose which manager’s notes a request may read?
nat-session cookie value that the client sends with the requestA preference lookup calls store.search with the namespace prefix (tenant_id,) and returns items belonging to other managers in the same tenant. The lookup must return only the requesting manager’s preferences, stored under (tenant_id, user_id, "preferences"). Which change fixes it?
query that ranks this manager’s items first(tenant_id, user_id, "preferences") namespacelimit to the manager’s item countthread_idReferences
- NVIDIA, Writing Per-User Functions
- NVIDIA, Memory in NVIDIA NeMo Agent Toolkit
- LangChain, Checkpointers
- LangChain, Stores
- Python Software Foundation, Synchronization Primitives
- Redis, Transactions