Assign retrieval ownership across authorized sources

Learning objective

Design multi-agent RAG with per-agent retrieval responsibility, source authorization, and cross-source query routing.

Core idea

For an architect designing multi-agent retrieval-augmented generation (RAG), more retrievers should mean clearer responsibilities and better evidence coverage. It should not mean every agent searches every source with the broadest available credential. Assign each specialist a retrieval domain, authorized sources, and a result contract. Give one coordinator responsibility for assembling evidence and deciding whether coverage is sufficient.

Retrieval responsibility, query routing, and authorization are separate controls. A planner can choose a useful source, but that choice does not grant access. A service identity can call a search endpoint, but that permission does not prove the end user can read every indexed document. Preserve these distinctions before evidence enters an agent’s context.

See 2.2.6 for memory isolation and consistency, 2.2.10 for ranking, and 2.2.12 for knowledge integration interfaces. This lesson connects those mechanisms into a cross-source retrieval design.

How it works

Give each retriever a bounded contract

Define a specialist by the evidence it owns. A product-documentation retriever returns supported technical behavior; a policy retriever returns governing conditions; an account-data retriever returns authorized customer-specific records. These are application roles, not built-in Foundry agent types.

The contract should carry the scoped question, permitted sources, trusted identity context, and evidence budget. Its result should include supporting text, source identifiers, document versions where applicable, and unresolved parts of the question. A narrative answer without these fields makes downstream validation and citation difficult. See 2.2.2 for bounded handoff contracts.

For an agent using the Foundry Azure AI Search tool, project_connection_id and index_name identify the retrieval target. The tool’s filter applies to all queries the agent makes to that index. This gives a concrete source-and-selection configuration for a specialist rather than relying only on instructions to avoid unrelated content.

The documented tool targets one index, and the search resource and Foundry agent must be in the same tenant. Cross-source retrieval therefore requires an explicit design beyond assuming one tool setting covers arbitrary indexes. A coordinator can call distinct retrieval integrations, or an Azure AI Search knowledge base can orchestrate multiple configured knowledge sources.

Enforce both service and source access

A service credential authorizes access to Azure AI Search. For the documented Foundry IQ integration, assigning Search Index Data Reader to the project’s managed identity grants read access to search indexes. It is not a substitute for a user’s document-level permissions.

Azure AI Search supports generally available security filters based on stored identity strings and trusted application query logic. The application resolves the caller’s permitted identifiers and constructs the filter. An agent-generated user or group value is not a trustworthy identity assertion. See 1.2.4 for the filter pattern; preserve the same protection across each retrieval path.

Native indexed ACL/RBAC enforcement is preview. For knowledge sources that require it, ingestionPermissionOptions ingests the permission metadata, and the request carries the user’s identity through x-ms-query-source-authorization. The retrieve documentation warns that indexed sources return unfiltered results when the user identity is omitted, and that missing ingestion permission configuration leaves results unfiltered regardless of the header. Check both prerequisites before sending a protected-source request.

Remote sources enforce permissions through their own documented integration. Remote SharePoint, for example, uses the caller’s token with its retrieval API rather than requiring content to be copied into a local search index. Tokens and headers are source-specific; the retrieve page documents a separate Work IQ authorization flow. Do not reuse one generic credential assumption across all connectors.

After retrieval, preserve authorization when agents exchange evidence. A specialist authorized to read a confidential source must not automatically send its full result to every agent. Restrict the receiving path and any persisted context according to the task’s actual readership.

Route questions to the sources that can answer them

Route by the information needed, not merely by which source has similar vocabulary. A question about technical capability belongs with product documentation; a question about contractual eligibility needs policy or contract evidence even if the technical manual mentions the same feature.

Azure AI Search agentic retrieval offers a concrete multi-source mechanism: a knowledge base references knowledge sources, and the application calls its retrieve action. LLM subquery planning needs preview low or medium retrieval reasoning effort; see 2.2.12 for the API versions. Select planning when the question requires decomposition across sources rather than assuming every retrieval call needs another planner.

For explicit source control, knowledgeSourceParams names sources and their request options. alwaysQuerySource=true requires a source to be queried. Starting with 2026-08-01-preview, neverQuerySource=true excludes a source for that request and overrides a stored always-query setting. These are routing controls, not permission grants. For search-index knowledge sources, filterAddOn adds an OData selection filter.

When authoritative policy evidence is mandatory, set failOnError=true for that source: if it fails, the preview retrieve request returns 502 Bad Gateway even when another source succeeds. For source types that support alwaysQuerySource, set both options to require consultation and successful retrieval. MCP server knowledge sources (preview) do not support alwaysQuerySource; their failOnError applies only when the source is selected. Do not treat these two controls as universally interchangeable or available.

Keep coverage and failures visible to the coordinator

Request includeActivity=true to obtain retrieval activity and use per-source includeReferences=true when evidence provenance is required. The activity array exposes source operations and timing; the references array connects grounding data to its origin. A reference’s activitySource links it to the producing activity, and docKey identifies an indexed document.

For preview retrieval, HTTP 206 Partial Content means at least one source succeeded and no failed source is marked failOnError; the activity array includes failed-source errors. A 502 Bad Gateway instead means every selected source failed or a required source failed, and failure details might appear only in the top-level error. Make the coordinator inspect that outcome instead of accepting every nonempty evidence set as a successful cross-source answer. An optional supplemental source can fail without invalidating a technical explanation; a failed mandatory contract source should prevent an eligibility conclusion.

Which sources failed decides the status code, and the coordinator inspects either outcome rather than accepting nonempty evidence.

Resolve disagreement by source authority and applicability in application policy. Preserve the competing passages and their versions rather than letting a synthesizer silently merge contradictory claims. Microsoft provides retrieval operations and evidence fields; the business rule for which source governs a decision remains yours.

Key distinctions and trade-offs

Specialist ownership versus knowledge-base planning: Specialists own application responsibilities. A knowledge base can decompose and retrieve across sources without becoming the business decision owner.

Routing versus authorization: Source selection changes where a query runs. Identity, permission metadata, and source enforcement determine what content it may return.

Nonempty evidence versus complete coverage: A result can contain relevant material while omitting a failed mandatory source. Coverage must be assessed against the question’s clauses.

Specialist separation improves accountability but adds coordination work. Shared knowledge-base retrieval simplifies source orchestration but still requires task-specific evidence validation. Use the smallest arrangement that keeps authority, coverage, and failure handling explicit.

Architecture / decision scenario

A customer asks whether a documented feature is available under its support contract. The coordinator assigns technical capability to a documentation specialist and eligibility to a policy specialist. Each receives the same trusted customer context but a different source scope and evidence contract.

The documentation path uses a one-index Search tool with product-version filtering. The policy path uses a preview knowledge-base request with protected-source ingestion configured and the user’s authorization header present. The contract source is marked alwaysQuerySource=true and failOnError=true; an unrelated source is excluded with neverQuerySource=true.

The coordinator receives evidence and source references from both specialists. It checks activity for source failures and compares policy versions before synthesizing a conclusion. If technical evidence succeeds but contract retrieval fails, it returns an incomplete eligibility outcome rather than guessing from the manual. The final answer cites the authorized governing passages, and no specialist’s broader credential becomes authority for the other path.

Common exam traps

• Giving every specialist unrestricted retrieval because they share a coordinator.
• Treating Search Index Data Reader as proof of end-user document access.
• Assuming missing user authorization headers fail closed for indexed knowledge sources.
• Confusing alwaysQuerySource with an authorization or mandatory-success setting.
• Assuming MCP server knowledge sources support alwaysQuerySource.
• Treating HTTP 206 Partial Content as complete evidence coverage.

What to remember

Assign responsibility before adding retrievers. Enforce service access and source permissions separately, route each question clause to suitable sources, and return provenance and source failures to the coordinator. A cross-source answer is complete only when its required evidence is both authorized and available.

Knowledge check

References

• Microsoft, Connect an Azure AI Search index to Foundry agents
• Microsoft, Document-level access control
• Microsoft, Agentic retrieval overview
• Microsoft, Query a knowledge base via API or MCP
• Microsoft, Connect agents to Foundry IQ knowledge bases