Make agent decisions inspectable and contestable

Learning objective

Design human-AI experience (HAX) controls for transparency, feedback, override, and accountability.

Core idea

Human control needs a visible action and an execution contract behind it. A person must understand what the agent proposes, provide feedback or correction, stop or reject consequential work, and identify who owns resolution when something goes wrong. A reassuring interface cannot provide these guarantees if the runtime continues independently of the person’s decision.

Human-AI experience (HAX) design connects these controls to the person’s task. Microsoft’s Azure Well-Architected guidance recommends the Microsoft Human-AI Experiences (HAX) Design Library as a source of principles and implementation guidelines. Use transparency, feedback, override, and accountability controls to apply that guidance to the interaction and its execution contract.

Microsoft’s responsible AI guidance treats transparency and accountability as lifecycle responsibilities. It recommends source disclosure, feedback that lets people contest decisions, auditable agent activities, and intervention controls. Design these together: transparency makes a decision inspectable, feedback makes it contestable, override changes execution, and accountability connects the outcome to a responsible owner.

See 1.1.6 for where to place checkpoints. Here the question is what the person experiences and how their response changes the system.

How it works

Show evidence and execution status

Explain the supported task, the limits of the agent’s role, and when a person remains responsible for a decision. Separate a suggested action from an executed action. The interface should distinguish proposed, awaiting approval, running, failed, and completed outcomes using application state tied to actual execution evidence.

Microsoft’s Azure Well-Architected responsible AI guidance recommends references to the source documents used for an answer and step-level logs for multi-agent workflows. Preserve source metadata through retrieval so the interface can display usable references. A generated explanation without supporting evidence does not prove that a cited record was retrieved or an external operation succeeded.

Present a concise explanation of the proposed action: target, arguments, expected effect, and important uncertainty. Let people inspect more detail when needed instead of showing every internal message by default. Microsoft’s guidance recommends gradual, minimally disruptive disclosure rather than overwhelming the person with information.

For a tool call, the runtime’s proposed arguments are the concrete review object. If the model says it plans to update one account but the arguments identify another, the interface must show the actual executable request. Do not make approval depend only on the model’s narrative.

Make feedback actionable

Feedback about answer quality differs from authorization to act. Microsoft’s guidance suggests concise controls such as thumbs up or down and aspect-specific ratings, and also requires a way to contest AI decisions. A positive answer rating should not approve a payment. A rejected recommendation should not disappear into an aggregate satisfaction score without a review path.

Capture enough application context to investigate: response or job identifier, the answer version, feedback category, and any correction the person supplies. These fields are an application design, not a Foundry feedback schema. Associate them with the execution record while minimizing retained personal data.

Route feedback to an owner and define its effect. A factual correction can request a revised answer. A contested business decision needs review by someone authorized to reconsider it. A runtime failure belongs to operational recovery. The person should receive an acknowledgement and understand whether execution paused, continued, or already completed.

Treat feedback as user data. Microsoft notes that collecting it affects storage architecture and requires privacy controls. Do not automatically persist every correction as long-term agent memory or training material. See 1.1.10 for memory admission and retention responsibilities.

Bind approval to the proposed call

For Foundry remote MCP tools, require_approval defaults to always; the documented values include always, never, and tool-specific lists. When approval is required, the response includes an mcp_approval_request output item describing the tool and arguments.

The application presents that request and returns an mcp_approval_response. Its approval_request_id identifies the approval request item, and approve carries the decision. The follow-up request uses previous_response_id to continue the original response. Keep these identifiers distinct: the approval item ID identifies the proposal; the response ID identifies the continuation.

This makes approval a specific runtime decision rather than a general “I trust this agent” preference. Application policy should require another review if the proposed operation changes materially. A corrected amount or target is a different proposal; do not reuse a human decision that covered different arguments.

Agent Framework function tools have their own approval request and response types (see 1.1.6), so they are not interchangeable with the MCP items.

Provide override and accountable recovery

Rejection before execution prevents the proposed effect. Cancellation during execution requires runtime and downstream cooperation. Reversal after execution requires a supported business operation. Design separate controls for these three moments rather than labeling them all “undo.” Rejection, cancellation, and reversal each act on a different execution state.

Microsoft’s responsible AI architecture guidance recommends circuit breakers, monitoring that can halt problematic execution, and human intervention that can override agent decisions. The application must connect a stop request to an enforced execution gate. It must also report effects that already occurred; stopping orchestration cannot by itself reverse an accepted external transaction.

Record the proposal, decision, authorized reviewer, execution outcome, and responsible operational owner. Record material model and configuration changes as well: Microsoft explicitly recommends auditable model selections, updates, hyperparameter changes, and data-processing changes. Do not record secrets or unnecessary personal content to create an audit trail.

Accountability remains with the workload team and business owner. A log can show what happened, but a person or team must own complaints, incident response, and remediation. Define an escalation path that remains usable when the agent cannot complete the task or the user disputes its outcome.

Key distinctions and trade-offs

Explanation versus evidence: An explanation makes an action understandable. Source metadata and execution records support claims about what actually happened.

Feedback versus approval: Feedback evaluates or contests an outcome. Approval authorizes a specific proposal to proceed. Use separate records and controls.

Reject versus stop versus reverse: Rejection occurs before execution, stopping limits further work, and reversal compensates for a completed effect. Their guarantees differ.

More disclosure can improve review but also burden people and expose sensitive data. More approval prompts can reduce autonomy but encourage habitual acceptance. Use concise, consequential review objects and meaningful escalation rather than measuring control by the number of dialogs.

Architecture / decision scenario

A billing assistant proposes a credit after retrieving policy and invoice evidence. The interface names the retrieved sources, shows the proposed amount and account, and reports that no credit has executed. The MCP response contains an mcp_approval_request; the application displays its arguments rather than relying on the assistant’s summary.

The customer reports that the invoice is incorrect. That feedback creates a correction record and a review task; it does not approve the credit. An authorized reviewer then rejects the proposal using an mcp_approval_response linked to the request item and original response. The system reports rejection and retains an appropriate decision record.

If a separate operation is already running, Stop blocks further application dispatch and initiates supported cancellation. The interface lists any confirmed effects and routes reversal requests to billing staff. The workload owner can reconstruct the model version, evidence, proposal, reviewer decision, and service result, then resolve the complaint. Transparency, feedback, override, and accountability each change a concrete part of this process.

Common exam traps

• Treating a thumbs-up rating as permission to execute a tool.
• Approving a generated summary without inspecting executable arguments.
• Confusing approval_request_id with previous_response_id.
• Promising that stopping an agent reverses completed transactions.
• Assuming an audit log assigns a human owner or resolves a contested decision.

What to remember

Show evidence and actual state, connect feedback to review, and bind approval to an identified proposal. Give rejection, cancellation, and reversal separate guarantees. Preserve enough evidence to investigate while assigning a responsible owner for decisions and recovery.

Knowledge check

References

• Microsoft, Responsible AI in Azure workloads
• Microsoft, Design foundations for agents
• Microsoft, Connect agents to MCP server endpoints
• Microsoft, Using function tools with human in the loop approvals