Content Safety for an Online-Gaming Community

Why it matters

A safety classifier calibrated for general use can be wrong for one community. This notebook shows why you must measure missed harms and overblocking separately.

What to practise

Configure a type: content_safety model with llama-3.1-nemotron-safety-guard-8b-v3, the content safety check input $model=content_safety flow, and a prompt listing NVIDIA’s 23-category taxonomy. Note why the notebook checks input rather than output. Run the evaluation over the gaming-chat subset and compare recall on toxic messages with the over-block rate per benign category. Then read which category, by S-code, causes the false positives.

How it fits

This puts 9.3.2’s distinction into numbers: a rail can catch every toxic message in the subset and still block ordinary trash-talk. Whether to tune the policy, change the classifier, or pursue safety training is the decision 9.3.3 frames.