Route hard cases to stronger model tiers

Learning objective

Route work across model tiers by difficulty, escalating hard cases to a more capable model.

Core idea

Tiered routing saves money only if the system can distinguish work the cheaper route handles reliably from work that needs more capability. The routing decision and its mistakes are part of the product you evaluate. Measuring each model separately does not prove the combined path is safe or economical.

Anthropic recommends sweeping one model across effort levels before adding a second model, then pricing the stronger model alone at low effort. A pair earns its complexity by beating those alternatives on your workload. See 2.1.1 for tier comparison and 2.1.3 for reasoning controls; this Bite owns the decision to move work between tiers.

How it works

Separate workload assignment from escalation

A model per workload routes by a known task category: classification can start on Haiku, while complex synthesis starts on Opus. Escalation routes within a workload: routine cases start cheaply, but a detectable hard case moves to Sonnet, Opus, or Fable after a specific condition.

In a client-controlled Messages integration, the implementing field is model. Your application chooses the approved model ID when assembling the next request. A route name in an application configuration is your own abstraction; it is not an API parameter that automatically detects difficulty.

An initial classifier can assign a tier before inference, or a validator can reject a first result and trigger another tier. These answer different questions. Classification predicts difficulty from the input; validation tests an observable result. A classifier adds its own latency and error cost. A validator may require an initial attempt that is subsequently discarded and billed.

Choose signals with a task-specific meaning. A reconciliation route might escalate when extracted line items do not sum to a supplied total. A coding route might escalate when a relevant executable test fails. Neither proves that every passing case is correct, so evaluate both the escalated cases and the cases the cheap route accepts.

Make the stronger attempt useful

Pass the original task, relevant source material, and the failed check to the stronger model. Label the initial answer as an unaccepted candidate. Sending only its confident summary can preserve the very omission that caused the failure. The second attempt must satisfy the same acceptance criteria; a higher model tier is not an acceptance signal.

Keep a bounded escalation path. For example, one initial Haiku attempt followed by one Sonnet attempt, with unresolved cases assigned to a named review queue. This is a proposed application policy, implemented by request assembly and attempt accounting around model, rather than a vendor-provided retry setting. Assign the service owner responsibility for changing that policy when the hard-case mix changes.

On that bounded path, a result the validator rejects gets one stronger attempt under the same acceptance criteria, and what is still unresolved goes to the review queue.

Distinguish difficulty escalation from transport recovery and safety outcomes. Anthropic’s beta server-side fallbacks feature handles safety-classifier refusals; it is not a general low-score or hard-task router. A response with stop_reason: "refusal" belongs to the documented refusal handling path. Don’t count that category as proof that the original task exceeds the model’s reasoning capacity.

Price the route, including its misses

For a two-stage route, a useful first estimate is initial-attempt cost plus escalation probability multiplied by second-attempt cost. Add routing and validation work to that estimate. Divide total spend by accepted outcomes when comparing candidates, so that unresolved cases do not disappear from the denominator.

Measure missed escalations as well as escalation volume. A cheap route can appear efficient because it accepts errors instead of detecting them. Conversely, a validator that escalates nearly everything pays for the cheap attempt without keeping enough work there to matter.

Anthropic’s advisor tool, a beta feature, implements a distinct form of escalation: a smaller executor retains the loop and consults a stronger model for difficult decisions. Consultation helps only when it occurs. A workflow that needs a complete replacement answer can use client-controlled rerouting instead; it does not need to become an advisor architecture. Delegation topology is covered in 1.4.2.

Key distinctions and trade-offs

Detection quality versus model quality: A stronger second model cannot repair a hard case that the router never sends to it. Measure false acceptance among first-stage passes separately from second-stage recovery.

Serial escalation versus direct assignment: Serial escalation pays the first attempt’s time and tokens on the hard tail. Direct assignment avoids that duplicate work when the input already identifies a difficult category reliably.

Model change versus effort change: For checkable outcomes, test the policy of starting at low effort and re-running failed checks on the same model at a higher output_config.effort. Anthropic documents this as re-running failures at higher effort. A two-tier route earns its additional request configurations and support constraints only by beating that escalation baseline on your workload.

Advisor versus replacement: An advisor supplies guidance while the executor continues. A replacement attempt returns the candidate that the application validates. They create different handoff and acceptance responsibilities.

Architecture / decision scenario

A financial-services team prepares transaction-reconciliation drafts. An overnight run has a 20-minute deadline, and the accounting owner requires fewer than 0.2% incorrect accepted reconciliations. The team proposes Haiku followed by Sonnet when a deterministic amount check fails. Another architect proposes direct Sonnet for every case.

In a hypothetical 10,000-case replay, the initial attempt costs $0.002 per case and 10% escalate at an additional $0.018 each. The two-stage model cost is $38, versus $180 for direct Sonnet; these are measured scenario costs, not list prices. The routed replay finishes in 14 minutes. But checking only the escalations reveals nothing about confidently wrong passes.

The accounting team audits first-stage passes against verified reconciliations and finds a 0.35% accepted-error rate: matching totals concealed incorrect account assignments. That fails the requirement despite the saving. The architect adds an account-assignment check and replays the combined route. Escalation rises to 18%, cost rises to $52.40, completion reaches 18 minutes, and accepted errors fall to 0.12%.

The accounting owner accepts the revised route. Direct Sonnet remains the baseline and the alternative if escalation growth breaks the deadline. The service owner monitors validator misses and the escalation fraction; a cost saving cannot override the accounting quality floor.

Common exam traps

• Measuring the stronger model while leaving router mistakes unmeasured.
• Treating the initial model’s confidence as a calibrated difficulty score.
• Counting only the successful second attempt’s cost.
• Adding another tier before comparing a single-model effort sweep.
• Using fallbacks as if it escalated arbitrary validation failures.
• Assuming a stronger answer bypasses the original acceptance checks.

What to remember

Route by an observable task distinction, and escalate through explicit model selection when the application detects a hard case. Evaluate the complete route, especially wrong first-stage passes. Keep the pair only while its accepted-result quality, deadline, and total cost beat the simpler alternatives.

Knowledge check

References

• Anthropic, Choosing the right model
• Anthropic, Models overview
• Anthropic, Optimizing for cost and intelligence
• Anthropic, Refusals and fallback