The SRE Incident Response Agent

Why it matters

An incident agent is most useful, and most dangerous, when it can change production. This notebook shows where investigation must end and approved remediation begin.

What to practise

Run a Claude Agent SDK agent against local Docker infrastructure through a stdio MCP server that wraps Prometheus queries, container logs and config edits. Break the API by setting DB_POOL_SIZE=1, then give a vague report with fixes forbidden, and watch the agent correlate error rates, latency, connections and logs to the config file. In a second run it edits the file, redeploys, verifies and writes a post-mortem. Two PreToolUse hooks block an out-of-range pool size before an edit or a redeploy. Notice that the investigation run keeps its write tools in allowed_tools and is held back only by its prompt.

How it fits

That last point is 7.3.4’s test: read-only access belongs in the tool set, not the prompt, and remediation needs explicit approval and pre-execution checks like these hooks. The notebook’s runbook skill is a start on 7.3.5’s rehearsed diagnostic procedures. It pins an earlier Claude model; the mechanism is unchanged.