Export agent traces to Foundry and Application Insights

Learning objective

Configure Foundry-connected OpenTelemetry/Application Insights tracing, token metrics, optional prompt-content capture, and correlation IDs; distinguish Foundry trace views from external and local trace trees.

Core idea

A visible trace tree proves that one viewer received some spans. It does not prove that Microsoft Foundry received them, that token metrics were exported, or that client and server operations share a trace. For architects integrating observability, configuration has four separate parts: instrumentation, exporter destination, identity and correlation, and content policy.

Foundry stores connected trace telemetry in Azure Monitor Application Insights. Its trace view is a product view over that telemetry, while an external backend or local collector can receive a different set of spans. Choose the destination deliberately and verify each required signal there.

Foundry tracing is generally available for prompt and hosted agents. Workflow and external agent tracing is preview. Foundry SDK client-side GenAI instrumentation is separately documented as experimental preview. These scopes matter even when all paths use OpenTelemetry.

How it works

Connect the project before instrumenting application code

In the Foundry project, Agents > Traces > Connect creates or connects an Application Insights resource. The alternative path is Manage > Project details > Connected resources > Add connection > Application Insights. Server-side tracing is the documented starting point: connecting the resource enables capture for supported Foundry-hosted execution without application code changes.

Query access is separate from connection setup. The tracing page specifies Log Analytics Reader on the connected resource for telemetry queries, plus Privileged Monitoring Data Reader when underlying Log Analytics tables are protected. A successful agent invocation does not prove the person inspecting telemetry has those permissions.

The Foundry trace view exposes ingested traces from the last 90 days. Application Insights storage follows the connected workspace’s retention and pricing configuration. Treat the portal’s viewing window and your telemetry retention policy as separate settings; see 3.3.2 for broader correlation records.

Instrument and export the code you own

For Python Foundry SDK client-side instrumentation, set AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING="true" before AIProjectInstrumentor().instrument(). The opt-in enables the experimental preview instrumentation. The SDK documents automatic instrumentation of Responses and Conversations calls; AZURE_TRACING_GEN_AI_INSTRUMENT_RESPONSES_API="false" disables that automatic path while retaining explicit custom spans.

Obtain the connected destination through project.telemetry.get_application_insights_connection_string() and pass it to configure_azure_monitor(connection_string=...). Add an application span with trace.get_tracer(...).start_as_current_span(...) around the business request so your surrounding work has an observable parent. These names describe the Python path. .NET uses the Azure.Experimental.EnableGenAITracing AppContext switch; JavaScript uses tracingOptions: { experimental: true } on AIProjectClient.

For Microsoft Agent Framework Python, the framework documents another integration path: configure Azure Monitor first, then call enable_instrumentation(enable_sensitive_data=False) to activate the framework’s telemetry code paths. configure_otel_providers() is the framework’s provider/exporter setup helper when it owns that configuration (see 3.3.1). Select one provider setup strategy deliberately; enabling instrumentation alone does not choose your storage backend.

Foundry’s framework integration also documents use_microsoft_opentelemetry() for Agent Framework agents hosted outside Foundry. Set enable_azure_monitor=True, supply the connected azure_monitor_connection_string, and configure instrumentation_options["agent-framework"] with enabled, agent_id, and agent_name. Foundry hosted agent server packages configure export and enrich spans with project and agent identity automatically. An external application must supply the corresponding routing and identity configuration explicitly.

Verify tokens as a distinct signal

Agent Framework emits invoke_agent <agent_name>, chat <model_name>, and execute_tool <function_name> spans (see 3.3.1). For chat operations it also emits the gen_ai.client.token.usage histogram and gen_ai.client.operation.duration histogram. Tool timing uses agent_framework.function.invocation.duration. A spans-only exporter does not establish that these metric instruments reached your backend.

Model-call spans can carry gen_ai.usage.input_tokens and gen_ai.usage.output_tokens attributes when usage is available. Those attributes explain a recorded call; the histogram supports aggregate measurement. Validate both the expected span attributes and the metric export path when your operations dashboard depends on tokens. Keep missing usage distinguishable from a measured zero.

Semantic conventions are another configuration dependency. Foundry’s overview identifies OpenTelemetry GenAI conventions as Development status. Agent Framework uses experimental conventions when OTEL_SEMCONV_STABILITY_OPT_IN is unset. When set, it selects convention behavior: a set value without gen_ai_latest_experimental selects v1.36 conventions; including that case-sensitive token selects experimental conventions. Pin and test your queries against the chosen package and convention behavior instead of assuming field names never change.

Choose content capture independently

Python Foundry SDK content recording uses OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="true" before instrumentation. It captures message contents, tool arguments, and model output. Leave recording disabled unless the environment’s privacy and retention requirements permit it. Metadata and token measurements often answer the operational question without complete prompts.

The SDK explicitly warns that this switch controls built-in traces only: functions instrumented with @trace_function record all parameters and return values regardless of it. Audit custom spans and decorators as well as the global setting. A disabled capture switch is not a guarantee that no application code emits content.

Agent Framework uses its own enable_sensitive_data option or ENABLE_SENSITIVE_DATA environment variable, whose default is false. Its sensitive data includes prompts, responses, function arguments, and results. Configure the relevant instrumentation’s setting rather than assuming one variable controls every library. See 1.2.7 for minimized evidence records.

Test correlation and destination separately

The Python Foundry SDK propagates W3C traceparent and tracestate headers for OpenAI clients acquired through get_openai_client() when tracing is enabled. AZURE_TRACING_GEN_AI_ENABLE_TRACE_CONTEXT_PROPAGATION controls this behavior; changes apply to clients obtained after the change. Preserve these identifiers through the supported transport, rather than replacing them with a conversation ID.

A trace ID groups related execution spans; a span ID identifies one operation. Response and conversation IDs locate service results and dialogue history, which Foundry also exposes from the Traces page. They support investigation but serve a different purpose from trace context. See 3.3.2 for joining business decisions across services.

Local Foundry Toolkit tracing uses an OTLP-compatible collector and does not require Application Insights. Local model and tool calls can still reach cloud services and incur charges. Likewise, exporting to an external OTLP backend produces that backend’s tree; it does not by itself create a Foundry trace view. Confirm ingestion into the project-connected Application Insights resource when Foundry visibility is a requirement.

Key distinctions and trade-offs

Instrumentation versus export: Instrumentation creates telemetry. Export sends each signal to a destination. An instrumented process can still export only to its console.

Server versus client visibility: Server-side capture explains managed execution. Client-side instrumentation adds your orchestration and application operations, with its own experimental preview scope.

Content versus correlation: Correlation needs identifiers and propagation, not complete prompts. Content capture can improve diagnosis while increasing access and retention obligations.

Central export improves production investigation but adds telemetry storage costs and permission requirements. Local capture shortens development feedback, while external tools can provide their own views. Choose complementary destinations only when each one serves a defined operational purpose.

Architecture / decision scenario

A Python agent application shows a complete local trace tree, but the operations team sees no matching Foundry trace and no token dashboard. The local OTLP collector is the only configured destination. Enabling more prompt capture would increase content exposure without repairing export.

The team connects Application Insights to the project and obtains its connection string through the project telemetry API. It selects one Azure Monitor provider setup, activates the relevant SDK and framework instrumentation, and verifies agent identity attributes for the externally hosted agent. It retains a root application span and checks traceparent propagation on a newly acquired OpenAI client.

A test request must appear in the connected Foundry view, carry input/output usage where reported, and produce gen_ai.client.token.usage in the configured metric destination. A second test confirms sensitive capture is disabled and audits custom decorators. The team labels external tracing preview and SDK GenAI instrumentation experimental preview, while retaining local collection for development.

Common exam traps

• Treating a local tree as proof of Foundry ingestion.
• Enabling instrumentation without configuring an exporter.
• Assuming a trace exporter also exports token histograms.
• Applying one library’s content switch to every instrumentation layer.
• Confusing conversation IDs with trace context.
• Applying prompt-agent tracing’s GA status to external agents or SDK instrumentation.

What to remember

Connect Application Insights, enable the correct instrumentation, configure each required export signal, and verify identity and correlation at the destination. Token metrics and prompt-content capture are separate choices. Foundry, external, and local trace viewers show what reaches their respective backends; they are not interchangeable proof of coverage.

Knowledge check

References

• Microsoft, Set up tracing for AI agents in Microsoft Foundry
• Microsoft, Add client-side tracing to Foundry agents
• Microsoft, Configure tracing for AI agent frameworks
• Microsoft, Observability
• Microsoft, Agent tracing overview