Rank use cases for production funding

Learning objective

Prioritize use cases and justify investment by baseline comparison, adoption assumptions, feasibility, risk, and time to production.

Core idea

An impressive demonstration does not establish that a use case deserves the next production budget. You need a credible improvement over a comparable baseline, enough adoption to realize it, and a delivery path the organization can operate within its risk tolerance.

Use an investment decision record to compare candidates. Attach the value map from 1.6.1, the workflow cost ledger from 1.6.2, and the service targets from 1.6.3. The decision record adds the assumptions, dependencies, evidence gates, and production date that determine whether those promises are achievable. The proof-of-value method in 1.1.4 tests a risky assumption; this lesson decides how that evidence changes portfolio priority and funding.

How it works

Compare against a credible alternative

A candidate’s baseline is the service it replaces or augments, with equivalent scope and acceptance criteria. Include the viable non-AI option: a process change, better search, or an existing deterministic integration might deliver the improvement with less investment. NIST AI RMF 1.0 MANAGE 2.1 explicitly includes viable non-AI alternatives when considering resources and risk reduction.

Write the comparison as a bounded claim: for a defined population, change a specific accepted outcome from its measured baseline to a target, while meeting quality and service constraints. “Save thirty minutes per employee” leaves the workflow, usage frequency, review burden, and baseline method unspecified.

Keep incremental benefits distinct from inherited benefits. If a search redesign already reduces handling time, do not credit all of that reduction to Claude. If the proposal adds coverage outside working hours, acknowledge that the manual baseline lacks that service; compare its incremental outcome and cost rather than pretending the two scopes are identical.

Expose adoption and benefit assumptions

Forecast eligible volume, adoption among eligible users, successful completion, and realized value per accepted outcome. A simple forecast multiplies those factors, but the record must state what each means. Adoption is actual use of the intended workflow, not licenses assigned or invitations sent.

Use base and downside cases. Lower adoption reduces benefit while leaving many integration and maintenance commitments unchanged. Poor completion increases retries and human workload. A favorable forecast that depends on near-universal adoption is weak until pilot behavior supports it.

Validate adoption with the people who do the work. Record whether the candidate fits their existing task, what approval remains, and what training or process change delivery requires. NIST MAP 1.1 calls for documenting users, expectations, assumptions, and deployment context; MAP 3.4 addresses operator proficiency. An adoption assumption therefore needs an evidence source and an owner, not a persuasive percentage chosen by the project team.

Do not monetize released time twice. If finance expects a cash saving, name the expense that falls. If the benefit is additional service from the same staff, use accepted output and opportunity value with an explicit sponsor assumption. The value map carries that distinction into the funding decision.

Test feasibility and risk before ranking

Assess feasibility at the workflow boundary: source-data availability and rights, access to the required systems, integration effort, evaluability, human capacity, and an operational owner. A good model response on hand-picked documents does not prove these dependencies.

Anthropic’s evaluation guidance calls for tests that mirror the real task distribution, including edge cases. Its cost guidance recommends production-weighted tasks, outcome checks, baseline comparisons across candidate configurations, and shadow evaluation before cutover. Use a representative evaluation result and a shadow-run report as evidence attachments rather than claiming feasibility from a demo.

Architectural complexity must earn its place. Anthropic’s building-effective-agents guidance recommends starting with the simplest solution and adding complexity when needed, noting that agents trade latency and cost for task performance. Use the pattern decision in 1.3.4 as an input. Do not award investment points merely because a candidate uses more agents or tools.

Risk is a gate as well as a comparison dimension. NIST MAP 1.5 documents organizational risk tolerance; MAP 3.2 documents monetary and non-monetary costs; MANAGE 1.2 prioritizes risk treatment by impact, likelihood, and available resources. Attach a risk-register entry naming the harm, affected people, mitigation, residual risk, and decision owner. A large projected benefit does not authorize crossing an unaccepted risk boundary.

Rank by deliverable value and revisit the decision

Time to production means time to an accepted, supported release, including access approvals, integration, evaluation, training, and ownership. Estimate it from named dependencies and their owners. Prototype duration alone omits the work that often determines the production date.

First remove candidates that fail non-negotiable feasibility or risk gates. Then compare deliverable value, downside economics, and time to usable benefit. A weighted score can summarize those comparisons, but keep its weights and underlying evidence visible. It cannot compensate for missing data access or a risk owner who has not accepted deployment.

Make funding conditional on an evidence gate and review date. MANAGE 1.1 asks whether the system achieves its intended purpose and whether development or deployment should proceed. Translate that into a record with continue, fix, pivot, and stop criteria. These are decision conventions for your delivery, not Anthropic API states. A pivot changes the target use case and requires a revised baseline and business case.

Key distinctions and trade-offs

Potential value versus realizable value: A large eligible population gives an upper bound. Adoption, completion, operational capacity, and launch timing determine what the organization can realize.

Rank versus permission: Relative priority chooses among acceptable candidates. A feasibility or risk gate decides whether a candidate is acceptable at all.

Prototype speed versus production readiness: A quick demonstration can reduce uncertainty without proving integration, operating ownership, or support capacity.

A smaller use case can win when it delivers verified benefit sooner and supplies reusable evaluation or integration assets. Count that enabling benefit only if the dependency is concrete and owned. Avoid inventing strategic value to rescue a candidate with weak direct evidence.

Architecture / decision scenario

A university has an illustrative $80,000 budget and eight weeks before enrollment opens. It compares a policy assistant, staff-assisted document checking, and an autonomous enrollment adviser. All figures are scenario assumptions.

The policy service handles 10,000 eligible cases per cycle. Its contracted staff baseline costs $2 per accepted case, or $20,000 per cycle, with the same quality and service scope. Finance confirms that each accepted automated resolution removes that $2 charge; cases not resolved automatically retain the contracted service. The assistant forecasts 60% adoption and 80% accepted completion among adopted cases, avoiding 4,800 staff cases and $9,600 of baseline cost.

Launch costs $24,000, leaving $56,000 of the budget uncommitted. A 1.6.2-style ledger projects $3,600 per cycle for inference, retrieval, tools, evaluation, incremental human review, and maintenance, including failed attempts. Remaining contracted case handling is additional. Net benefit is $9,600 minus $3,600, or $6,000 per cycle: four cycles repay launch cost. At 30% adoption, avoided staff cost falls to $4,800 and operating cost to $3,000, leaving $1,800 net benefit and payback of about 13.3 cycles. Launch cost is counted once in payback, not again in operating cost.

Public policy content and checkable outcomes support a six-week release, with an owned staff handoff. Document checking has stronger potential savings but a ten-week student-record access dependency, missing the enrollment window. Autonomous advice lacks agreed completion criteria and accepted risk ownership.

The sponsor requires payback within six cycles, so accepts the base case and rejects the downside case. The architect recommends a $6,000 pilot within the $24,000 launch allocation, holding the remaining $18,000 behind an adoption, quality, and cost gate. Representative evaluation and shadow-run evidence must support at least $4,000 net benefit per cycle, the six-cycle payback threshold, before full funding proceeds.

At the agreed review, a diagnosable content defect can justify a bounded fix and re-evaluation. A sustained $1,800 net benefit with no credible improvement path stops further funding. Value concentrated in staff handoff summaries can justify a pivot, with a revised baseline and ledger. The decision uses realized outcomes and the funding threshold rather than demonstration quality.

Common exam traps

• Ranking candidates by novelty, enthusiasm, or number of agents.
• Forecasting benefits from all eligible users without an adoption assumption.
• Comparing a constrained pilot with an unrelated manual baseline.
• Treating a weighted score as permission to bypass a risk or access gate.
• Equating prototype completion with a supported production release.
• Ignoring the non-AI alternative or the downside economics.
• Continuing funding without an evidence gate, decision owner, and review date.

What to remember

Fund a use case that improves a comparable baseline, survives plausible adoption assumptions, and can reach supported production within accepted risk. Keep the decision record inspectable: evidence, costs, dependencies, owners, and conditions for continuing or stopping. Prioritization is a judgment about deliverable value, not technical ambition.

Knowledge check

References

• NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0)
• Anthropic, Define success criteria and build evaluations
• Anthropic, Optimizing for cost and intelligence
• Anthropic, Building effective agents