Test policy independently of model compliance

Learning objective

Separate permission-based tool access, pre-tool business-policy enforcement, and prompt guidance; test that prohibited side effects are blocked regardless of model compliance.

Core idea

A tool can be available to the agent, permitted for use, and still inappropriate for a particular transaction. A refund tool might be authorized for customer support while a proposed $700 refund violates the application’s automatic-payment ceiling. Treat tool access and transaction policy as separate decisions.

Prompt guidance serves a third purpose: it helps Claude choose the right workflow and explain outcomes. It does not replace code that rejects a prohibited operation. The architecture must remain safe when the model proposes the forbidden call anyway.

In the Claude Agent SDK, use permission configuration to define access, a PreToolUse hook to enforce the per-call business predicate, and systemPrompt in TypeScript or system_prompt in Python to guide behavior. Verify the boundary by observing execution, not by scoring how politely the final answer describes the rule.

How it works

Assign one responsibility to each layer

The three layers answer different questions:

Python’s equivalent top-level permission options are allowed_tools, disallowed_tools, and permission_mode; its prompt option is system_prompt. Hook output retains identifiers such as hookSpecificOutput and permissionDecision in both languages.

Use prompt instructions to explain the automatic-refund ceiling and the human-review alternative. Enforce the ceiling through the hook decision described in 1.5.1. Normalize lookup evidence separately through PostToolUse; see 1.5.2.

Understand what an allow list actually does

allowedTools adds permission allow rules; it does not remove every unlisted tool. An unlisted tool remains available and follows the remaining permission flow. Therefore, allowedTools: ["Read"] does not by itself establish a read-only agent.

A bare-name disallowedTools entry removes that tool’s definition from the model’s request. A scoped rule such as Bash(rm *) keeps Bash available and denies calls matching that pattern as written.

Pairing allowedTools with permissionMode: "dontAsk" denies other calls that would require a prompt. It still does not remove tools or prohibit every unlisted operation: calls requiring no approval can run. To remove an unwanted capability entirely, name it in disallowedTools rather than relying on absence from the allow list.

Removing Bash does not impose a monetary ceiling on an allowed refund tool.

Keep the policy gate separate from access approval

See 1.4.2 for the permission evaluation order and why canUseTool is not a mandatory gate; see 1.5.1 for the PreToolUse denial output. A permitted tool still needs a policy check on each governed operation.

An access confirmation and an application-specific business approval have different meanings. If a denied transaction requires review, verify that review through the application’s separate workflow rather than interpreting a permission dialog as approval of the refund.

Test the execution boundary directly

Use an instrumented implementation of the application-owned refund tool. Record whether its handler is entered and whether it writes a refund record. Exercise the hook through the deployed SDK configuration as well as testing the predicate in isolation. A correct predicate with an incorrect matcher does not guard the real tool.

For this application’s USD ceiling of $500 per operation, drive these cases through the test harness. Amounts use integer cents, and the passing case assumes that all other prerequisites and access checks succeed:

The direct-call cases challenge the predicate and registration even when the model follows its prompt. The model-directed case exercises the agent’s actual proposed action. A session in which Claude never requests the refund does not demonstrate that interception works.

Run the bypassPermissions case only in a controlled test environment. For the parallel-hook and timeout semantics behind the final two rows, see 1.5.1.

Key distinctions and trade-offs

Availability versus permission versus policy: Removing a tool, requiring approval to use it, and rejecting particular arguments are separate controls.

Behavior quality versus execution safety: Prompt tests evaluate whether Claude selects the intended workflow. Interception tests evaluate whether forbidden calls are stopped even when that selection fails.

Proportional enforcement versus prompt guidance: Decide which layer carries a rule by the cost of one failure, not by how often the model complies. If a single wrong call is costly or irreversible (it moves money, changes identity data, or breaks a compliance obligation), enforce the rule in a PreToolUse hook or the tool’s dispatch path. If a deviation is cheap and recoverable, such as a skipped courtesy step that a later turn repairs, prompt guidance can carry it alone. Anthropic’s Claude Code best practices call CLAUDE.md instructions advisory and hooks deterministic, and reserve hooks for actions that must happen every time with zero exceptions.

A guarded route versus every side effect: A hook protects the tool calls it matches. If the same payment can be issued through another enabled tool, that route needs equivalent enforcement or removal. State the guarantee in terms of the actual guarded execution paths.

Architecture / decision scenario

A support agent can investigate a customer case, issue an eligible refund, or request human review. Its prompt explains the $500 automatic-refund ceiling. Permission rules authorize the application’s refund tool and exclude unnecessary alternate execution capabilities. Its PreToolUse callback evaluates the monetary policy before the tool handler begins. A missed end-of-case summary is cheap to repair, so the prompt alone asks for one.

The test harness submits a $700 refund even though the prompt prohibits it. The hook returns "deny"; no refund handler runs and no refund record exists. Auto-approving that tool does not change the outcome. The human-review instruction appears in the denial reason, but the team verifies a separate review request before claiming escalation succeeded.

The same harness confirms that an otherwise eligible $500 request executes once. It also tries an alternate payment tool and finds that the intended matcher would not cover it. The team removes that capability or adds the same policy check before accepting the design.

Common exam traps

• Reading allowedTools with permissionMode: "dontAsk" as removing unlisted tools.
• Treating a scoped disallowedTools rule such as Bash(rm *) as removing Bash.
• Confusing permission confirmation with business approval.
• Answering a rule that must hold every time with few-shot examples, a threshold in the tool description, or a larger model; each only raises the odds of compliance.
• Accepting a compliant final answer as proof that interception ran.
• Testing only the policy predicate and omitting matcher registration.
• Claiming all payment routes are protected after testing only one tool.
• Leaving an irreversible payment rule in the prompt because the model usually follows it, or moving every recoverable preference into hooks; the cost of a single failure decides.

What to remember

Permissions control access, PreToolUse enforces the per-operation rule, and the prompt guides workflow selection. Force prohibited calls in tests and require zero side effects. Put a rule in code wherever one failure is costly or irreversible.

Knowledge check

References

• Anthropic, Configure permissions
• Anthropic, Control execution with hooks
• Anthropic, TypeScript SDK reference
• Anthropic, Python SDK reference
• Anthropic, Best practices for Claude Code