Separate session state from shared memory

Learning objective

Select persistence components for session state, shared team state, and long-term semantic memory.

Core idea

Persist a value according to what it means and who may change it. A conversation is a record of interaction. A checkpoint is execution state. A shared team record coordinates several agents. Long-term semantic memory retains useful knowledge across sessions. These values can describe the same task without being interchangeable.

Start with lifetime, authority, and access scope. Then select the component. A search result shouldn’t become the authoritative approval record, and a persisted conversation doesn’t automatically provide a resumable workflow checkpoint. The fact that a value survives a restart says nothing about whether it belongs to the right tenant or contains only necessary data.

See 1.2.3 for principal assignments and 1.2.4 for compliance controls. Here, decide which record survives which failure and who can recover it.

How it works

Persist sessions without confusing them with conversations

Hosted agents distinguish sessions, conversations, and a durable state store. A session identifies logical compute state, including $HOME and uploaded /files content. Foundry restores that state when it provisions compute after an idle period. Each session is isolated from other sessions.

The configured idle timeout is 2–60 minutes per agent version, with a 15-minute default. It controls compute deprovisioning, not immediate deletion of persisted state. The platform permanently deletes a session after 30 days of inactivity. A design requiring longer retention must select a suitable separate store rather than treating the sandbox as a permanent archive.

A session’s compute comes and goes with idle time, while its state persists until 30 days of inactivity delete the session.

A conversation ID identifies durable history of messages, tool calls, and responses. That history persists independently of compute state. With the Responses protocol, Foundry manages conversation history and associates a session with the conversation. With Invocations, the client manages the session ID and your code manages history. Choosing a hosted runtime doesn’t make both protocols’ history behavior identical.

Use session persistence for working files needed by subsequent turns. Use conversation persistence for interaction continuity. Keep resumable business execution state in explicit checkpoints rather than assuming a model can reconstruct the exact workflow position from a chat transcript.

Select a durable checkpoint component

Foundry’s durable state store is preview and available only to hosted agents. It stores keyed JSON items independently of container crashes, restarts, and idle eviction. In Python, FoundryStateStore.get_or_create binds a client to a caller-chosen store name; set_item, get_item, delete_item, and list_keys operate on its items.

An item can contain up to 1 MB of serialized JSON. Put large binaries in blob storage and retain a reference in the item. Store names are stable identities and can’t be renamed after creation. A default 30-day store-level idle window removes items; writes renew the window and reads don’t. Configure nonexpiring behavior at creation only when the retention requirement justifies it.

User isolation is also a creation choice: user_isolation=True. On container protocol 2.0.0, x-agent-foundry-call-id identifies the caller, and the SDK forwards that header so the store can resolve the acting user. Local container runs don’t support this user-isolation boundary; verify it with a deployed hosted agent.

Don’t encode user IDs in store names. The documentation warns that names appear in operational surfaces and that application-generated names aren’t verified identity. Use non-user scope such as a workflow or run in the store name, and the platform’s user isolation where appropriate.

Every item has an ETag. For mutable read-modify-write state, use If-Match to prevent a lost update. An append-only checkpoint can instead use a fresh key for each save. The state store persists your JSON; it doesn’t interpret your business state machine or decide which transition is valid.

Give shared team state an explicit owner

Separate state shared inside one workflow run from authoritative records shared across runs. Agent Framework workflow state lets executors exchange common values: Python uses ctx.set_state() and ctx.get_state() with consistent keys. In .NET, use QueueStateUpdateAsync and ReadStateAsync with the same non-null scopeName; null selects the executor’s private default scope. In .NET, a writer sees its update immediately, while other executors see it in the next superstep.

Graph workflows persist resumable execution through superstep-boundary checkpoints. For a Python workflow hosted in Foundry, CheckpointStoreProvider from agent_framework_foundry_hosting connects checkpoint persistence to ResponsesHostServer through checkpoint_store_provider. The Python agent-framework-foundry-hosting package is prerelease. Host the native workflow through workflow=; the documentation deprecates wrapping it through agent= because shared in-memory workflow state can cross users or conversations.

Use application-owned Cosmos DB records for cross-run case status, approvals, and artifacts shared by independently authorized agents. Define permitted writers and valid transitions. A /TenantId partition provides logical separation with shared throughput; separate accounts enable account-level controls such as customer-managed keys. Foundry’s bring-your-own Cosmos DB conversation storage serves platform history, a separate purpose from your domain records. A common hosted state-store name alone doesn’t establish cross-agent access.

Distill semantic memory separately

Foundry Memory and the Memory Store API are preview. They provide managed long-term extraction, consolidation, and retrieval across sessions. Supported types include user profile memory, chat summary memory, and procedural memory. The store uses compatible chat and embedding model deployments; an embedding alone doesn’t establish extraction or lifecycle behavior.

In Python, project_client.beta.memory_stores.create creates a store. MemoryStoreDefaultOptions includes user_profile_enabled, chat_summary_enabled, procedural_memory_enabled, and default_ttl_seconds. Configure retained categories and defaults deliberately. These controls apply to memory entries, not every original conversation or external copy.

For the memory search tool, scope="{{$userId}}" enables automatic per-user resolution. The service first uses x-memory-user-id if present; otherwise it uses the caller’s Entra tenant ID and object ID. A trusted proxy must derive the header from authenticated end-user context. Forwarding an arbitrary client value would make the application’s own isolation design unsafe.

Low-level memory APIs require an explicit scope in every request; they don’t perform that automatic identity extraction. A static scope intentionally shares one memory collection, so don’t use it for unrelated users by accident.

The documented quotas are 100 scopes per store and 10,000 memories per scope; search and update each allow 1,000 requests per minute. Memory is region-limited: the documented list includes East US 2 and Australia East; check the Memory reference’s Region availability list for the target region. Memory stores don’t support VNet integration. If preview status, region, scope capacity, or private networking conflicts with the workload, select application-owned persistence and retrieval.

Key distinctions and trade-offs

Select by the record’s purpose:

Managed persistence reduces infrastructure work but brings documented lifecycle and scope constraints. Application-owned stores provide design freedom while requiring you to implement access, update rules, and retention. Semantic relevance and business authority remain separate regardless of the physical database.

Architecture / decision scenario

A support team contains a conversational agent, diagnostics agent, and approval agent. Responses stores the user conversation. A hosted session retains uploaded diagnostic files while active and across idle restoration; permanent evidence goes to governed Azure Storage.

Within one run, diagnostics and approval executors exchange state through consistent Python state keys or a shared .NET scopeName. Graph checkpoints resume at superstep boundaries. For a Python hosted workflow, accept the prerelease hosting package and configure CheckpointStoreProvider; keep the native workflow request-aware. For separate preview state-store items, preserve the 1 MB limit, call-ID user isolation, and ETag updates.

All three agents use an application-owned Cosmos DB case record for shared status. Each has a defined write responsibility; the approval field isn’t reconstructed from a conversation summary. Tenant partitioning or separate accounts follows the required isolation level.

For future sessions, Memory can retain approved preferences in a per-user scope with configured TTL, separate from authoritative case outcomes. Check the target region against the documented availability list; exceeding scope capacity or requiring VNet-only access changes that choice. Each store now serves its own recovery and authority contract.

Common exam traps

• Equating idle compute eviction with immediate session deletion.
• Assuming Invocations supplies Responses-managed conversation history.
• Treating a summary as an authoritative approval or checkpoint.
• Sharing a store name without evaluating identity and user isolation.
• Assuming low-level Memory APIs infer scope from the caller.
• Treating preview Memory as unlimited or VNet-integrated storage.

What to remember

Choose persistence by purpose, lifetime, authority, and scope. Separate interaction history, working files, checkpoints, shared team records, and semantic memory. Confirm each component’s limits and deletion behavior, and make recovery preserve the same authorization boundary as the original operation.

Knowledge check

References

• Microsoft, Hosted agents in Foundry Agent Service
• Microsoft, Durable state store for Microsoft Foundry hosted agents (preview)
• Microsoft, Use your own resources in the Foundry Agent Service
• Microsoft, Multitenancy and Azure Cosmos DB
• Microsoft, What is Memory?
• Microsoft, Create and use memory in Foundry Agent Service
• Microsoft, Microsoft Agent Framework Workflows — State
• Microsoft, Workflow concepts
• Microsoft, Foundry Hosted Agents