Route history, episodes, and facts to stores

Learning objective

Assign working context, conversation history, and durable episodic and semantic knowledge to suitable stores.

Core idea

Choose a store by what the information means, who can reuse it, and how long it remains useful. Working context is the information supplied to the model for one decision. Conversation history records exchanges within a thread. Episodic memory preserves experiences; semantic memory preserves facts. These are information roles, not four mandatory database products.

A conversation can survive a process restart without becoming cross-session knowledge. LangGraph makes this distinction explicit: a checkpointer persists thread-scoped state, while a store holds application-defined information across threads. NeMo Agent Toolkit provides a memory subsystem for history, preferences, and long-term information. Its common interface does not remove the architect’s responsibility to decide which role each record serves.

How it works

Separate the prompt from its sources

Working context is an assembled view: the active request, applicable instructions, selected history, tool observations, and retrieved memories. Keep its token budget separate from storage capacity. LangGraph’s memory documentation explains that full histories can exceed the model’s context window; even histories that fit can add latency and cost and distract the model with irrelevant content.

Storing a year of messages therefore does not require sending a year of messages on every turn. Preserve the history needed by the application, then select the portion appropriate to this decision. Buffer truncation and rolling-summary implementation belong to 5.1.1; here, decide which source retains the complete conversation and which representation enters inference.

Give conversation history a thread

In LangGraph, short-term memory is part of graph state, persisted through a checkpointer. The invocation’s configurable.thread_id identifies the thread whose state continues. This is suitable for a support conversation that must remember its exchanges across requests. It also separates that conversation from another conversation by the same person. NeMo Agent Toolkit names its conversation key conversation_id; nat run --conversation_id sets it for CLI testing, with support depending on the memory backend.

Use persistent storage when continuity must survive worker loss. A Python object holding a message list cannot satisfy that requirement on its own. Conversely, persisting graph state does not make every field suitable for reuse in another task. 5.4.1 separates execution state from retained task knowledge; do not treat a checkpoint as a reusable user profile.

Classify durable knowledge before choosing an index

Semantic memory answers questions such as “What language does this customer prefer?” It can be a profile document or a collection of facts. LangGraph stores long-term memories as JSON documents under a namespace and a distinct key. store.put(namespace, key, value) writes a document; store.get(namespace, key) retrieves a known item. A profile lookup with a known identifier does not inherently require embeddings.

Episodic memory answers “What happened during the previous incident?” Preserve an experience with its outcome and enough context to judge whether it applies. LangGraph describes few-shot examples drawn from past sequences as one use of episodic memory. A resolved incident can become an example for another incident, but its past action is not proof that the same action is authorized or appropriate now.

Either kind can use semantic search. The word semantic in semantic memory describes factual knowledge; in semantic search it describes a retrieval technique. An embedding index can retrieve an episode, while an exact-key document store can retrieve a semantic fact. Choose the retrieval path after choosing the information’s role. See 5.1.2 and 5.1.3 for extraction, updates, and retrieval mechanics.

Working context receives selected history and retrieved memories; the complete conversation stays in the source that retains it.

Connect the stores to NVIDIA’s workflow

NeMo Agent Toolkit declares providers under memory. A provider’s _type selects its implementation; the documented Zep example uses nat.plugins.zep_cloud/zep_memory. The nvidia-nat-zep-cloud plugin supplies that provider. Provider choice includes deployment and data-handling requirements, not only retrieval quality.

The workflow._type: auto_memory_agent wrapper connects inner_agent_name, memory_name, and llm_name. It retrieves context before invoking the wrapped agent and can store user and assistant messages. retrieve_memory_for_every_response, save_user_messages_to_memory, and save_ai_messages_to_memory control those behaviors and each defaults to true.

Automatic capture is a policy choice: enable it when the destination is intended to hold those messages. If only validated facts belong in a durable profile, disable the wrapper’s message writes. NVIDIA’s provider example defines _type: add_memory and _type: get_memory functions over a mem0_memory provider, each referencing it through memory: saas_memory. These tools let the agent write preference facts and retrieve them deliberately, but NVIDIA warns that LLMs may forget memory-tool calls, making capture inconsistent. The wrapper instead schedules capture automatically. Neither approach establishes fact quality; see 5.1.2.

NVIDIA documents MemoryEditor.add_items, search, and remove_items as the provider extension interface, and MemoryItem carries conversation, metadata, user_id, and optional textual memory. Use that seam to implement the chosen storage roles rather than assume automatic message capture performs fact validation.

For narrower controls, the wrapper passes add_params to memory_editor.add_items() and search_params to memory_editor.search(). Its Zep example uses add_params.ignore_roles: ["assistant"] to exclude assistant roles from graph memory, and search_params.top_k to cap retrieved memories. Zep’s search_params.mode selects "basic" or "summary". Role exclusion controls what enters graph memory; it does not validate the remaining facts.

Key distinctions and trade-offs

Use these distinctions when selecting stores:

One physical database can hold several roles if its access paths and lifecycle policies distinguish them. Separate services can isolate operational responsibilities, but add network calls and administration. A vector database is a retrieval option, not a universal memory architecture. A database-backed LangGraph store is appropriate for production persistence; its InMemoryStore example is for development and testing.

Architecture / decision scenario

A manufacturer’s maintenance assistant serves 600 engineers. The delivery team sets a 3-second response target and a 6,000-token working-context budget. A typical active conversation contains 18,000 tokens; including it whole violates that budget before manuals or observations enter the prompt. The tempting proposal is to place every message in one vector index and use its nearest matches as the only memory.

The application architect rejects that as the sole store design. Thread-scoped checkpointed history retains the ordered conversation; a selected view supplies working context within 6,000 tokens. Durable episodic records preserve resolved equipment incidents with outcomes. Semantic records preserve confirmed equipment facts and engineer preferences under known keys. Retrieval adds only relevant experiences or facts to the active view, rather than replacing the thread history with approximate matches.

The workflow owner connects the chosen provider through memory_name and decides whether the automatic wrapper’s message-write flags fit that destination. A profile-only provider does not receive unfiltered conversation writes. The operations owner measures the added memory-read latency against the 3-second target; if it consumes the budget, the team reduces search_params.top_k and evaluates Zep’s mode: "basic" against "summary", measuring recall and injected tokens as well as latency. All numbers are scenario requirements, not toolkit limits.

Common exam traps

• Equating short-term memory with volatile storage: thread-scoped state can be persisted.
• Treating every stored message as a durable, validated fact.
• Confusing semantic memory with vector similarity search.
• Using a checkpoint as the only cross-thread profile store.
• Assuming auto_memory_agent capture establishes the truth of a stored response.
• Assuming add_memory tool writes happen on every turn.

What to remember

Assign roles before products: assemble working context, persist thread history, and retain experiences and facts through deliberate cross-session stores. Choose exact lookup or semantic retrieval for the access pattern. NeMo Agent Toolkit’s provider interface and wrapper connect those decisions to execution; they do not make the decisions for you.

Knowledge check

References

• LangChain, Memory overview
• LangChain, Persistence
• LangChain, Stores
• NVIDIA, Memory in NVIDIA NeMo Agent Toolkit
• NVIDIA, Configure the Automatic Memory Wrapper Agent
• NVIDIA, Adding a Memory Provider