Classification with Claude: Insurance Support Ticket Classifier

Why it matters

Ticket classification is a classic first job to give Claude. This cookbook shows why that choice is justified by measured accuracy against labeled data, not by a good-looking demo.

What to practise

Set up the evaluation first: a labeled test set, a random baseline, accuracy and a confusion matrix. Then build three classifiers in turn. The first uses category definitions only, with the response prefilled with <category>, stop_sequences set to </category> and temperature set to 0. The second adds the five most similar training tickets from a vector database as few-shot examples. The third adds a scratchpad for reasoning before the label. Watch which categories the confusion matrix shows being confused, and how each change moves them.

How it fits

The accuracy target is 1.1.2’s acceptance contract. The exercise runs on claude-haiku-4-5; Claude 4.6 models reject a prefilled final assistant turn with a 400 error, so on them the same control is the output_config.format structured output that 1.1.2 uses. The step from a baseline to a measured gain is 1.1.4’s proof of value: a falsifiable claim tested before production.