AnthropicClaude Certified Architect – Professional2026-107.3 Support debugging and operational issue resolution
Rehearse runbooks against confirmed incident causes
Completed by the knowledge check
Learning objective
Capture diagnostic procedures as maintained runbooks or Skills and turn root-cause findings into rollback steps, regression tests, and documentation.
Core idea
An incident transcript records one investigation. A maintained diagnostic procedure must let the next operator recognize the same failure, distinguish it from similar symptoms, and verify recovery without reconstructing the original conversation. Converting the transcript into a skill is useful only if that conversion preserves those decisions.
Assume the choice between a task-specific skill and universal project guidance is already established. The Professional addition is maintaining the procedure as an operational artifact: named owner, tested prerequisites, verified branches, and changes tied to confirmed causes. See 6.4.4 for general runbook ownership and 4.6.4 for the wider incident-to-release cycle.
How it works
Extract a procedure rather than preserving the conversation
Start from the confirmed failing condition and the check that established it. For example, “the monitoring server used a relative script path and failed outside the repository root” supports a launch-path diagnostic and reproduction. “The assistant could not query monitoring” is only a symptom; it does not justify a permanent credential-reset step.
Give the procedure a trigger, required access, initial observations, decision branches, stop conditions, and expected result. Use the concrete diagnostic commands from 7.3.1 when they answer the question: /mcp for connection state, /hooks for registration, /permissions for effective rules, and claude --debug-file <path> for hook execution. Avoid instructing the operator to run every diagnostic after every alert.
Each branch should explain what the observation permits next. A failed connection can lead to path or credential inspection. A working connection with zero tools leads to server discovery diagnostics. Neither observation authorizes disabling a production deny rule. Capture the distinction that prevented the incident from becoming an access-policy change.
Anthropic’s best practices recommend verifiable outcomes and showing the executed check’s result. Apply that requirement to the runbook: “server connects and its required query succeeds from both supported launch directories” is stronger than “restart Claude and confirm it looks normal.”
Keep one maintained source of procedure detail
Keep the diagnostic entry point in .claude/skills/<name>/SKILL.md tied to the maintained procedure and its helper scripts. Reference a bundled helper as ${CLAUDE_SKILL_DIR}/scripts/diagnose.sh: ${CLAUDE_SKILL_DIR} resolves to the skill directory in both the body and allowed-tools Bash rules. For example, allowed-tools: Bash(${CLAUDE_SKILL_DIR}/scripts/diagnose.sh *) pre-approves that helper invocation rather than arbitrary shell commands. Version and review the helper with the skill, including its arguments and backend access.
Use an ordinary runbook when operators need the same procedure outside Claude Code, including during an assistant outage. If the skill wraps that runbook, have it point to the maintained detail rather than copying a second checklist that drifts independently. This is a team documentation design, not an automatic synchronization feature.
Version the relevant procedure and helper revisions with the service configuration. Record the service owner, the last rehearsal result, and the component versions it was exercised against in the team’s runbook metadata. These are application-maintained fields. A procedure referring to an executable that moved or a permission that no longer exists is broken even if its prose remains accurate.
Keep diagnostic invocation separate from remediation authority
A diagnostic skill can use ordinary invocation when its description matches the investigation task. If a separate remediation procedure must start only when a person chooses, disable-model-invocation: true prevents Claude from invoking that skill on its own. That controls initiation; it does not approve the requested production operation.
allowed-tools pre-approves calls during the invoking turn; it does not restrict which tools exist. To remove mutation tools from Claude’s pool while the skill is active, use disallowed-tools, for example disallowed-tools: [Edit, Write]. The restriction clears when you send your next message, so it is not a persistent access policy. Neither that turn-scoped restriction nor a narrow helper approval replaces the backend controls in 7.3.4; another tool such as Bash could still provide a mutation route.
Avoid embedding broad write approvals in a diagnostic skill for convenience. A reader should be able to stop with “cause supported, remediation requires the incident commander” and still have completed the diagnostic procedure successfully. Human approval and the tested safety hook apply when the production operation is actually requested.
Turn the cause into three concrete changes
A confirmed incident should produce distinct artifacts, each addressing a different recurrence risk:
| Artifact | What to preserve | Verification |
|---|---|---|
| Rollback procedure | Known-good target, preconditions, operator authority, stop condition, recovery check | Rehearse reversal and measure the stated recovery criterion |
| Regression test or evaluation | Minimal failing fixture and the condition the repair changes | Demonstrate failure before the repair and success afterward |
| Documentation change | Diagnostic branch, exact command or setting, cause, and limits of the fix | Have another operator follow it without the original author |
See 4.6.4 for selecting deterministic tests versus behavior evaluations, reproducing the failed outcome, and protecting unaffected neighboring cases.
Rehearse the failure, not only the happy path
Run the procedure in an authorized nonproduction environment with the diagnostic dependency missing, the relevant credential denied, or the policy script unavailable. Use the branch that corresponds to the confirmed fault. Record what the next operator can conclude and when they must stop.
For hook-related procedures, a successful policy rejection and a crashed hook are different outcomes. Preserve onFailure: "block" and its required deployed version where the design depends on blocking command-hook failures. For API incidents, preserve the error message and headers that separate a temporary rate limit from a spend cap. Reusing a generic “retry three times” branch would erase the lesson of the incident.
Key distinctions and trade-offs
A broad runbook covers more symptoms but becomes difficult to navigate under pressure. A narrowly triggered procedure is quicker to verify but needs a clear route from the initial symptom. Keep the entry point short and link to the branch the observed condition selects.
Automation reduces repeated query work, while maintenance adds rehearsal cost. Fund that cost through the owning team’s release process. A skill that nobody retests after an integration change transfers the investigation burden to the next on-call engineer.
Architecture / decision scenario
An education platform’s on-call rotation has four engineers and a twenty-minute recovery target. A release changes a tool-response adapter, which then discards valid enrollment records. Rolling back restores service, and a fixed-response reproduction identifies the parser fault. The team proposes saving the entire incident conversation as a skill.
The service owner rejects the transcript dump. The maintained runbook first compares the backend response with the delivered tool_result, names the known-good adapter revision, and assigns rollback approval to the on-call lead. A helper fixture reproduces the discarded-record case, while a valid empty-response case protects legitimate “no enrollment” behavior. The repair must pass both.
A diagnostic skill points to that runbook and fixture without granting write tools. A separate manual-only remediation skill describes the rollback request; production execution still uses the approval path. A second engineer rehearses the diagnostic branch in twelve minutes and verifies the rollback health check, while a missing fixture produces a stop and escalation rather than an unsupported diagnosis. The owner records the rehearsal with the procedure revision and updates it when the adapter changes again.
Common exam traps
- Saving an incident chat as a maintained runbook without extracting decisions.
- Treating successful rollback as a complete root-cause explanation.
- Testing only that the repair succeeds, without reproducing the original failure.
- Treating
allowed-toolsas a read-only restriction. - Treating manual skill invocation as approval of its production actions.
What to remember
Preserve the confirmed cause in a repeatable diagnostic branch, an executable rollback, and a targeted regression check. Keep one maintained procedure, rehearse it without its author, and retain independent authorization for remediation. A skill distributes a procedure; ownership and verification keep it useful.
Knowledge check
After an incident, a team turns its investigation into a diagnostic skill for the on-call rotation. The next operator must recognize the same failure and tell it apart from similar symptoms. What should the skill’s procedure be built from?
A diagnostic skill must keep the Edit and Write tools out of Claude’s reach during the turn that invokes it. The author set allowed-tools: Read Grep, yet Claude still edits a file in that turn. Which frontmatter change does what the author intended?
disable-model-invocation: true in the skillallowed-tools field to Read alone insteaddescriptiondisallowed-tools to [Edit, Write] in the skillallowed-tools pre-approves tools for the invoking turn and restricts nothing, while disallowed-tools removes tools from Claude’s pool while the skill is active. That restriction clears at the next message, so it is not an access policy, and disable-model-invocation only controls who starts the skill.A release broke an adapter that discarded valid enrollment records, and the fix is ready. Which regression check gives the strongest evidence that the fix addresses the confirmed cause?
A team adds a manual-only rollback skill with disable-model-invocation: true. An engineer argues that because only a person can start it, the rollback it requests needs no further approval. What is wrong with the argument?
allowed-tools grant to carry approval into its callsdisable-model-invocation: true stops Claude from invoking the skill on its own, which governs initiation only. The production operation the skill requests still goes through its own approval path and safety controls. A tool grant, a prompt instruction, or blanket approval for future runs does not establish human approval of the specific production action.On-call engineers must be able to follow a diagnostic procedure during a Claude Code outage, and the team wants only one maintained version of its detail. How should the diagnostic skill relate to the procedure?
References
- Anthropic, Extend Claude with skills
- Anthropic, Best practices for Claude Code
- Anthropic, Debug your configuration
- Anthropic, Hooks reference
- Anthropic, Define success criteria and build evaluations