AnthropicClaude Certified Architect – Professional2026-102.3 Apply prompt engineering techniques (zero-shot, few-shot, chain-of-thought)
Curate examples for production decision coverage
Completed by the knowledge check
Learning objective
Design few-shot examples that are diverse, representative, and cover edge cases without over-anchoring.
Core idea
Few-shot demonstrations already give you a way to clarify ambiguous choices through inputs, desired outputs, and short explanations. At professional level, your responsibility is to decide which demonstrations represent the organization’s policy, who approves them, and whether their improvement survives production variation.
Treat the example set as part of the deployed prompt. A demonstration can encode an unintended convention as readily as an intended one. Anthropic recommends examples that mirror the use case, cover edge cases, and vary enough to avoid unintended patterns. The architect’s decision is which consequential distinctions deserve scarce prompt space.
How it works
Give each example a production purpose
For each candidate, record the unresolved distinction it teaches, the applicable policy version, and the expected result. The policy owner approves that coverage obligation before the prompt owner allocates space to the demonstration.
Anthropic recommends 3–5 examples for best results generally; this is guidance, not an API maximum or a completeness guarantee. A compact set needs an explicit selection rationale. Five repetitive successes do not become representative because they fit the recommendation.
Use a coverage table to review the set. For an illustrative healthcare administration workflow, these are possible coverage obligations, not clinical rules:
| Decision boundary | What an approved demonstration establishes | What a held-out case changes |
|---|---|---|
| Missing information | Ask for a required identifier rather than invent it | Document type and wording |
| Conflicting records | Refer a material conflict for staff review | Which record supplies the conflict |
| Resolved correction | Accept a clearly superseded administrative value | Source order and correction phrasing |
| Ordinary agreement | Return the supported administrative result | Site, layout, and incidental details |
The policy owner approves the expected choices. The prompt owner makes sure the input actually supports each result. An example with an unjustified label teaches the wrong rule even if its output is perfectly formatted.
Separate representation from frequency
A representative set covers relevant decisions; it need not reproduce traffic percentages exactly. A rare but consequential conflict can deserve one of four demonstrations. However, a set containing only conflicts leaves the ordinary agreement case unexplained and can encourage unnecessary escalation.
Use the task’s real distribution for the main evaluation and a separate edge-case slice for the costly mistakes. Anthropic’s evaluation guidance calls for both real-world distribution and edge cases. Report both results so an apparent improvement on conflicts does not hide widespread false alarms on routine requests.
Over-anchoring is detectable when the same decision changes with an incidental feature. If every escalation demonstration comes from one hospital, test the same conflict with a different hospital name. If every accepted result uses the same document layout, test a valid result in another layout. Hold the decisive facts steady while varying the surface. The result should follow policy, not familiarity.
Review the set as one deployed artifact
Version the instructions and examples together in the release record, and review affected examples together whenever policy changes; see 2.2.5 for prompt releases and regression checks. The operational question is which exact prompt produced a decision, not which directory happened to contain plausible samples that day. Assign a domain owner to the labels and a prompt owner to deployment and testing. This is an implementation of the documented relevance and diversity guidance, not an Anthropic configuration feature.
Do not automatically promote a production correction into the prompt. First ask whether it exposes a missing rule, a missing fact, or a distinction that needs demonstration. Add the corrected case to development tests, then select a demonstration only if it fills a coverage obligation. Preserve a separate unseen test for that distinction.
Compare the added input tokens and maintenance work with the measured benefit; see 2.3.4. Consolidate incident cases around transferable distinctions rather than letting every correction expand the deployed prompt.
Key distinctions and trade-offs
Example-set size versus recurring cost: Each added demonstration consumes input tokens on every request that carries it and adds a label to re-approve when policy changes. The architect weighs that recurring expense against the coverage it buys; see 2.3.4.
Edge coverage versus excessive escalation: Rare failures deserve attention, but an all-failure set can misrepresent the intended behavior. Measure routine false alarms alongside edge-case recall.
Architecture / decision scenario
A healthcare scheduling network uses Claude to reconcile administrative referral details. Staff can review at most 4% of cases, and the business owner requires 99% detection of material conflicts in its labeled edge-case set. A colleague proposes pasting twenty staff escalations into the prompt.
The architect rejects that set because it contains no accepted or resolved-correction cases, and most samples share one clinic’s template. Four approved examples instead cover the boundaries in the table. The policy owner validates the labels, and the prompt owner keeps them inside <examples> with one <example> per case.
On illustrative held-out measurements, the all-escalation set detects 99.4% of material conflicts but refers 11% of routine traffic. The curated set detects 99.1% and refers 3.5%. Layout and clinic-name variations retain the same decisions. The curated set satisfies both requirements; the apparently stronger conflict score alone would exceed staff capacity.
The architect selects four examples and records the reason for each. When the administrative correction policy changes, the policy owner revises the affected label, and the prompt owner reruns both evaluation slices before release. No production correction enters the deployed set without that review.
Common exam traps
- Calling an all-escalation set representative because every sample came from production.
- Counting changed names as coverage of different decision boundaries.
- Treating the 3–5 recommendation as a required count or maximum.
- Promoting every correction into the prompt and grading on those same cases.
- Optimizing edge recall while ignoring the resulting human-review load.
What to remember
Own the example set as a policy-bearing prompt artifact. Select distinct, approved boundaries, vary incidental features, include ordinary behavior, and test transfer on unseen cases. Keep the recurring token and review costs justified by measured gains.
Knowledge check
A healthcare scheduling network uses Claude to reconcile administrative referral details, and staff can review at most 4% of traffic. A colleague proposes pasting twenty recent staff escalations into the prompt as examples. All twenty are material-conflict cases, and most share one clinic’s document template. Which example set should the architect choose?
A few-shot prompt’s escalation demonstrations all come from one hospital. The team suspects that the model escalates partly because of that hospital’s name rather than the conflict policy. Which evaluation tests that suspicion directly?
Staff correct a production decision in which Claude accepted a superseded administrative value. The prompt owner wants the mistake not to recur. What should happen first?
A claims-intake policy has six distinct consequential boundaries that written definitions leave ambiguous, and the policy owner has approved one demonstration for each. A reviewer objects to keeping a sixth example. What should decide whether it stays?
References
- Anthropic, Prompting best practices
- Anthropic, Define success criteria and build evaluations