Limit telemetry to its diagnostic purpose

Learning objective

Balance telemetry fidelity against data sensitivity, cardinality, storage cost, and retention.

Core idea

Telemetry is another data store with its own readers, copies, cost, and lifecycle. A prompt that tells an assistant not to reveal personal information does not remove personal information from the customer’s input or a tool result copied into logs. Decide what observations you need before enabling payload export.

For an architect, the artifact is a telemetry data specification: which fields leave the runtime, where content is processed, which dimensions reach metrics, what traces are retained, who can read them, and when each copy expires. Anthropic documents structural defaults and content switches; your organization owns its collection purpose and retention decision.

How it works

Start with structural observations

The Agent SDK uses its Claude Code subprocess to export telemetry. Anthropic describes this telemetry as structural by default: durations, model names, and tool names are recorded, while content the agent reads and writes is not recorded by default. Token counts depend on the underlying API returning usage data.

Agent SDK trace export is beta; see 3.4.2 for its enablement and scope. Keep content switches unset until the receiving pipeline is approved for the data. The important variables are distinct:

OTEL_LOG_RAW_API_BODIES=1 enables inline bodies; file:<dir> writes untruncated bodies to disk and exports a body_ref reference. Anthropic states that enabling raw bodies implies consent to what the other three switches reveal. A local file is therefore another sensitive copy, not an escape from the collection decision.

Inline tool output and OTEL_LOG_RAW_API_BODIES=1 bodies are truncated at 60 KB by default. CLAUDE_CODE_OTEL_CONTENT_MAX_LENGTH configures the limit on Claude Code v2.1.214 or later; file:<dir> bodies remain untruncated. Raw bodies redact extended-thinking content, but still contain the other conversation content. Avoid relying on truncation as redaction. A short value can still contain a complete secret or identifier. Prefer omitting a field to collecting it and hoping the payload limit removes the sensitive part.

Bound metric cardinality

A metric label creates separate series for its values and combinations. Request IDs, session IDs, and per-user dimensions can multiply series far beyond the number of routes or models. Keep high-detail correlation in diagnostic records and use bounded categories for routine metrics. Enabling OTEL_LOG_TOOL_DETAILS=1 can also increase series count by exposing real names on cost and token metrics; this attribution behavior requires v2.1.273 or later.

Claude Code documents OTEL_METRICS_INCLUDE_SESSION_ID=false and OTEL_METRICS_INCLUDE_ACCOUNT_UUID=false to remove those dimensions from metrics. It also provides OTEL_METRICS_INCLUDE_RESOURCE_ATTRIBUTES=false to keep custom resource attributes out of metric datapoint labels. Otherwise, custom keys from OTEL_RESOURCE_ATTRIBUTES become labels on every metric series.

Check what a switch changes beyond the intended chart. The Agent SDK observability page says a falsy OTEL_METRICS_INCLUDE_SESSION_ID also causes Claude Code to omit session.id from spans. If session timelines are required, weigh that loss against the series budget and preserve an approved application correlation mechanism. Do not assume every cardinality setting affects only metrics.

Use a controlled release cohort rather than an unlimited collection of prompt hashes as a metric dimension. Retain exact prompt revision references on traces. This lets dashboards compare active releases without keeping a permanent series for every experimental prompt.

Choose fidelity by failure and cost

Metrics summarize populations; retained traces explain selected executions. The source research article demonstrates diagnosing decision patterns and interaction structures without monitoring individual conversation contents. Structural tracing is therefore a meaningful operating choice when content collection is unacceptable.

When selecting traces in your own telemetry pipeline, distinguish head sampling, decided early without inspecting the whole trace, from tail sampling, decided using all or most spans. Head sampling is cheaper to decide early but can discard the particular rare failure later reported by a user. Tail sampling can prioritize known failures but requires a stateful component to buffer spans until trace completion, with a defined decision window.

Neither strategy guarantees preservation of unknown quality failures. An unsupported answer may have no error flag at trace completion, and delayed customer feedback may arrive after a tail decision. Keep a random baseline, record selection probabilities, and decide how delayed review joins to retained metadata. Do not claim that retaining every flagged trace retains every incorrect answer.

Compute population metrics before trace selection if those metrics must represent all requests. A failure-heavy retained sample is useful for investigation and unsuitable as an unadjusted estimate of overall error rate. These are application pipeline choices, not SDK sampling defaults.

Minimize content before it reaches storage

If a diagnostic question needs content, specify which fields are necessary and remove identifiers before export where possible. Collector-side redaction centralizes rules but still sends raw data to the collector and its processing buffers. The collector must then be inside the approved sensitive-data boundary. Redaction in the emitting process reduces that exposure but requires every producer to apply the rule consistently.

Keep content in a restricted store with access granted for the stated purpose. Put an opaque content reference on the structural record rather than copying payloads into several backends. Pseudonymous identifiers can still link activity to people through a lookup; constrain that lookup separately.

Define retention per store and purpose, including inline events, local raw-body files, content stores, indexes, and backups under your organization’s process. A reference that expires later than its target should show unavailable content rather than silently resolving to a different artifact. The operational owner verifies expiry and access; privacy and compliance owners approve the handling requirements. This is a collection design, not a claim that any fixed duration establishes compliance.

Key distinctions and trade-offs

Structural versus harmless: Paths, tool names, and identifiers can reveal sensitive activity even without full text. Review the actual exported fields.

Redaction versus truncation: Redaction removes selected information. Truncation limits length and leaves sensitive prefixes intact.

Sampling versus retention: Sampling decides which records enter the retained population. Retention determines how long those selected records and their copies remain available.

More fidelity supports deeper diagnosis but increases exposure and cost. A shorter retention window reduces accumulation but can prevent investigation of late complaints. Record that accepted limitation and preserve useful non-content aggregates for longer when policy permits.

Architecture / decision scenario

A healthcare scheduling service processes one million requests per month with eight spans per request. Its observability allocation permits 20 GB monthly, and its approved diagnostic design allows redacted payloads for seven days and structural records for 30 days. These are scenario constraints, not vendor defaults or regulatory durations.

At an assumed 2 KB per structural span, retaining all eight million spans costs roughly 16 GB before indexing and overhead. Adding an assumed 10 KB payload to each request adds another 10 GB. The proposal to enable raw API bodies everywhere already exceeds the allocation before replicas or indexes.

The architect retains aggregate metrics across all requests, limits metric dimensions, and samples structural traces with a random baseline plus flagged executions. Content remains opt-in for approved diagnostics, redacted before export, and stored separately with seven-day expiry. Local raw-body files are excluded from routine operation. The platform owner measures actual ingest and index overhead; the privacy owner reviews access and expiry. A late complaint can still be investigated structurally, with the accepted limitation that its payload may have expired.

Common exam traps

• Treating a prompt instruction as a logging privacy control.
• Calling truncation redaction or local raw-body files harmless.
• Putting request or user IDs on every metric series.
• Assuming tail sampling retains unrecognized quality failures.
• Estimating population rates from a failure-selected trace sample.
• Setting retention on one backend while ignoring other payload copies.

What to remember

Collect the minimum detail that answers the diagnostic question. Use documented content and cardinality controls, preserve sampling context, and specify retention across every copy. Fidelity, sensitivity, series count, and storage cost are one architectural trade-off with named operational owners.

Knowledge check

References

• Anthropic, Observability with OpenTelemetry
• Anthropic, Monitoring usage
• Anthropic, How we built our multi-agent research system
• OpenTelemetry, Sampling