Match API recovery to the failure

Learning objective

Debug API integration errors — rate limits, overload, timeouts, malformed requests — with retries, backoff, and fallbacks.

Core idea

The operational question is why recovery is failing despite a retry policy. A transient failure, a spend cap, a malformed conversation, and a client timeout can all appear as an unsuccessful request, but require different interventions. Preserve the response classification and the attempt history before choosing another route.

Assume local recovery already has one owner and a bounded budget. This lesson adds diagnosis of the Claude API path: distinguish traffic pressure from persistent constraints, reconcile SDK behavior with the service deadline, and decide when a fallback is defensible. See 4.6.1 for the full call-log schema.

How it works

Read the error and its context together

For the Claude API, log HTTP status, error.type, sanitized error.message, and request-id. The error body also carries request_id; Python response objects expose _request_id. Include the identifier when escalating a specific request to Anthropic support. These names describe this API path; do not assume another provider returns identical errors.

Do not count all 429s as a single capacity symptom. Requests per minute, input tokens per minute, and output tokens per minute are separate limiting dimensions. A sharp usage increase can trigger acceleration limits; gradually ramp traffic rather than repeatedly replaying the burst.

When retry-after is present, it specifies seconds to wait. A usage-tier monthly spend-cap 429 has no such header and persists until access resumes. The documented remedies are a higher limit or waiting for the cap to reset at 00 UTC on the first day of the next month. Escalate to the budget owner instead of increasing retry counts. Other missing-header cases still require their message to be inspected; absence alone is not a universal spend-cap detector.

Reconcile the retry configuration with the deadline

In Python, inspect the deployed max_retries and timeout, including request-level overrides through client.with_options(...). The SDK’s default two retries and ten-minute timeout are not a suitable service policy merely because they exist. A live endpoint with a short deadline needs explicit settings and a measured total recovery budget.

For a diagnostic reproduction, max_retries=0 exposes the first failure without SDK retries obscuring its timing. If the application deliberately owns scheduling, keep that setting and implement its bounded exponential backoff there. Otherwise, return recovery ownership to the SDK. Its transient retries use exponential backoff and honor retry-after when present. Record the effective policy so an operator can explain a slow success as well as a final exception.

Use randomized delay in a custom application scheduler to spread simultaneous callers, but never schedule a retry earlier than the service’s required wait. Before another attempt, account for delay, execution time, and result validation. A count limit does not establish that enough wall-clock time remains. A timeout can leave the caller uncertain about completion; repeated inference must not cause the application to repeat an external side effect blindly.

Repair malformed conversations at their source

A 400 following a tool turn often points to the adapter rather than model quality. Inspect the exact assembled history. For client tools, tool_result.tool_use_id must identify the preceding tool_use.id, and the tool-result message must immediately follow the assistant’s tool-use message. Put tool_result blocks first in that user message.

Fix the serializer and reproduce the failure with sanitized history. A prompt instruction to “return valid messages” cannot repair application code that inserts another message between a tool request and its result. An unchanged retry repeats the defect; a different model can merely make the broken branch less frequent. See 7.3.3 for separating that integration defect from a behavior regression.

Distinguish overload fallback from refusal fallback

An application can select a preapproved alternative by changing the target model and using the appropriate adapter. That is application routing, and it needs prior evaluation for task quality, latency, processing location, and tool compatibility. Spend only the remaining request budget. If neither path can meet acceptance, return an explicit unavailable or deferred outcome.

The Claude API’s server-side fallbacks: "default" feature is different. It is beta, requires anthropic-beta: server-side-fallback-2026-07-01, and retries safety-classifier refusals using recommended fallback models where available. It is not a general switch for recovering HTTP 529 overload. It is unavailable on Message Batches, Amazon Bedrock, Google Cloud, and Microsoft Foundry. Claude Haiku 5.5 has no server-side fallback; use client-side fallback with the SDK middleware or implement the retry in application code.

Also distinguish a failed stream from a failed initial request. The API can send an error after HTTP 200. Mark already-delivered partial output incomplete; do not silently splice another model’s output onto it. Streaming can help long-running requests avoid idle-connection failures, but the consumer still needs mid-stream error handling.

Key distinctions and trade-offs

More recovery attempts can lower returned transient-error rates while raising latency, traffic, and cost. Compare user-visible completion within the deadline with the attempts spent obtaining it. An apparently healthy endpoint whose retries consume its deadline has little room for fallback.

A retry preserves the route. A fallback changes a tested behavior and deployment path. A repair changes an invalid precondition. These are separate operational decisions with different owners: service operations, model-routing owners, and integration developers.

Architecture / decision scenario

A retail support endpoint has an eight-second application deadline. Its team proposes three application retries around a default Python client, then a fallback model. A failure sample instead shows three cohorts: 429s with retry-after: 12, 529s, and 400s immediately after parallel tool responses.

The service owner rejects the blanket loop. The twelve-second wait cannot fit the eight-second request, so that cohort receives the approved deferred outcome rather than an early retry. For 529s, the owner configures one recovery layer and reserves time for one evaluated fallback only when the remaining budget permits it. The routing owner confirms that fallbacks: "default" would not solve overload.

The integration developer fixes tool-result ordering for the 400 cohort and adds a serializer regression case. If a later 429 message identifies a tier spend cap without retry-after, the budget owner handles it. Success means each cohort receives a justified terminal outcome; merely reducing errors by hiding them behind another model would leave the defects unresolved.

Common exam traps

• Treating every 429 as a short-lived rate limit.
• Sleeping beyond the caller’s deadline because retries remain.
• Retrying a malformed tool history unchanged.
• Using refusal-specific fallbacks as an overload failover switch.
• Treating HTTP 200 as proof that a stream completed.

What to remember

Classify the cause before tuning recovery. Inspect the message, headers, request assembly, and effective SDK configuration. Bound retries and approved fallback by one deadline, and route persistent spending or serialization defects to their owners.

Knowledge check

References

• Anthropic, Claude API errors
• Anthropic, Rate limits
• Anthropic, Python SDK
• Anthropic, Handle tool calls
• Anthropic, Refusals and fallback