Allocate agent spending to its consumers

Learning objective

Track usage quotas, metering, cost allocation, and tenant chargebacks.

Core idea

For architects operating shared agent platforms, four questions require different records: how much traffic can a deployment admit, how much work did a tenant consume, what did Azure bill, and how should your organization allocate that bill? A tokens-per-minute quota answers the first question. It does not establish a tenant’s monthly cost or an invoice-ready charge.

Build a chain from trusted tenant identity to measured usage to financial reconciliation. Preserve the distinction between operational estimates and billable meters. A platform can enforce a tenant allowance in real time while finance settles the final allocation from Azure Cost Management records.

See 3.2.2 for deployment throttling and retries, and 3.3.6 for per-request execution budgets. This lesson owns the attribution and accounting decisions that span many requests.

How it works

Track capacity at its documented scope

Azure OpenAI in Microsoft Foundry Models assigns quota to a subscription by region, model, and deployment type, in units of tokens per minute (TPM). You allocate that capacity to deployments. Deployments also receive requests-per-minute (RPM) limits whose relationship to TPM varies by model; do not use a universal conversion ratio.

Maintain a capacity inventory with subscription, region, deployment type, model, deployment, assigned capacity, and observed consumption. When a tenant experiences throttling, first identify which shared deployment it used. A tenant entitlement does not reserve deployment capacity unless your routing and allocation design make that reservation real.

Quota allocation and billed usage have different arithmetic. Allocated TPM is a rate-limit assignment, not a count of tokens purchased or consumed. Operational rate-limit accounting can use estimates that differ from billing; see 3.2.2. Increasing deployment capacity changes admission, not your internal decision about who pays.

Meter tenants at the gateway

For supported LLM APIs routed through Azure API Management, llm-token-limit enforces per-key token rates and fixed-period quotas. Its identifiers include counter-key, tokens-per-minute, token-quota, and token-quota-period, whose values include Hourly, Daily, Weekly, Monthly, and Yearly. The policy returns HTTP 429 when the token rate is exceeded and HTTP 403 when the period quota is exceeded.

Derive counter-key from authenticated tenant identity or a validated API Management subscription-to-tenant mapping. Do not trust an arbitrary caller-supplied tenant header for chargeback. If one tenant owns several subscriptions, consolidate them deliberately rather than letting each subscription create an unrelated allowance.

The policy supports OpenAI Chat Completions and Responses schemas, Google Vertex AI, and Anthropic Messages in API Management v2 tiers. Its listed service tiers exclude Consumption. It tracks token usage independently at each gateway, including regional and workspace gateways. Therefore, a monthly quota on two gateways is not automatically one global tenant quota. Use a shared application ledger when your commercial contract requires that global property.

estimate-prompt-tokens controls preflight estimation. Without estimation, excess usage can be detected after the backend responds; concurrent requests can temporarily overshoot even with the policy. Treat this as an enforcement constraint when designing a strict prepaid allowance, rather than promising a perfect currency stop from a token counter.

Separate metric visibility from accounting completeness

llm-emit-token-metric sends LLM token custom metrics to Application Insights. Configure dimension elements with name and value, such as a stable tenant group and deployment. Provider-dependent metrics can include total, prompt, and completion tokens; cached, reasoning, thinking, and other categories are documented as preview. Do not assume every endpoint exposes every category.

The policy supports at most five custom dimensions. API Management limits each dimension to 100 unique values and each metric namespace to 1,000 active time series; additional dimension values or series are silently discarded. A tenant identifier for thousands of tenants therefore makes a poor complete financial ledger. Use bounded operational dimensions, and preserve detailed tenant usage in application-owned accounting records.

Streaming adds another completeness risk: interrupted streams can produce inaccurate captured counts. For OpenAI models that omit usage counts by default, especially in streaming responses, request counts with include_usage=true in the API request. Keep an explicit unknown or incomplete usage state rather than converting missing usage to zero. A sampled trace tree or aggregate dashboard can explain a cost spike, but it cannot certify that every chargeable invocation was counted.

Reconcile metering with billed cost

Microsoft Foundry’s cost documentation distinguishes estimated portal costs from Cost Management and invoiced charges used for financial reconciliation. Token billing units and rates vary by model, deployment type, and meter. Record input and output usage separately and retain any supported additional categories. Apply the price schedule associated with the actual meter and period, rather than multiplying all tokens by one remembered rate.

Include non-inference spending in the allocation plan. Foundry solutions can consume separately billed Azure services, and fine-tuned deployments can incur hosting charges even when unused. Document which costs are directly attributable, which are shared, and which remain platform overhead. HTTP status alone does not determine billing; reconcile with meter records.

Foundry project-level cost attribution is preview. It automatically adds a project tag to underlying usage so you can filter Cost Management by project. Support covers Models sold by Azure, including Azure OpenAI, and excludes models served through Azure Marketplace. A project becomes a tenant attribution unit only if your tenancy design assigns it that meaning. One shared project does not separate all its tenants by itself.

Allocate shared cost without changing Azure’s invoice

Cost Management cost allocation rules redistribute source subscription, resource-group, or tag costs to target subscriptions, resource groups, or tags. The feature supports EA, MCA-E, and MCA-online agreements under its documented billing-account permissions. Allocation changes cost analysis, not the Azure invoice or billing responsibility; your organization performs the chargeback.

Use Custom allocation percentages for an agreed allocation model; they must total 100%. Prefilled percentages remain fixed until you update the rule, so a usage-based policy needs a deliberate refresh process. Preserve the measured tenant shares and rule version that justify each period’s allocation. Export cost data to storage for downstream finance processing. The costAllocationRuleName column identifies allocated rows; filter them out when reconciling the original billed subscription invoice.

Cost Management budgets notify stakeholders when spending approaches thresholds. Budget notifications can invoke an action group to start custom automation, which requires additional development. They are not an Azure OpenAI hard spending limit. Pair them with operational quota enforcement when the platform must refuse further work.

Trusted tenant identity feeds two separate paths: the gateway refuses work in real time, while accounting records settle the chargeback.

Key distinctions and trade-offs

Quota versus meter: Quota limits admission. A meter records billable consumption under a specific pricing model. Unused quota is not evidence of zero hosting cost.

Attribution versus chargeback: Attribution connects usage or cost to a consumer. Chargeback applies your organization’s financial policy to that evidence. Showback displays responsibility without transferring internal charges.

Dedicated projects simplify supported project-level attribution but do not cover every shared service or model channel. Shared deployments improve utilization but require trusted tenant records and an explicit shared-cost policy. Choose the allocation unit from your tenancy and billing contract.

Architecture / decision scenario

A platform serves 500 tenants through shared Azure OpenAI deployments in two regions. Finance needs monthly chargeback, while operations needs per-tenant admission control. The team inventories deployment TPM and RPM separately from each tenant’s monthly token entitlement.

API Management derives counter-key from a validated tenant mapping and sets token-quota-period="Monthly". Because the gateways count independently, the application maintains the cross-region entitlement ledger. llm-emit-token-metric uses bounded service-tier dimensions for alerts; detailed tenant records remain outside the custom metric series that could discard values.

At month-end, finance reconciles those records with exported Cost Management meters. Tenants with dedicated projects use the preview project tag for supported Azure-sold models; shared-project and Marketplace spending follows the documented internal allocation method. A versioned 100% allocation rule distributes agreed shared overhead. Budget alerts warn about trends, while runtime controls refuse work; the internal chargeback does not alter Azure’s invoice.

Common exam traps

• Treating allocated TPM as prepaid tokens or tenant spend.
• Assuming a token quota aggregates across regional gateways.
• Using untrusted tenant headers as accounting authority.
• Expecting high-cardinality custom metrics to retain every tenant.
• Applying preview project attribution to Marketplace models.
• Treating a Cost Management budget as an automatic hard stop.

What to remember

Track capacity, token usage, billed meters, and allocated charges as separate records. Bind tenant attribution to trusted identity, account for gateway-local quota enforcement and incomplete telemetry, and reconcile financial totals with Cost Management. State the preview and model scope of project attribution and the internal nature of chargeback.

Knowledge check

References

• Microsoft, Manage Azure OpenAI in Microsoft Foundry Models quota
• Microsoft, Azure API Management policy reference - llm-token-limit
• Microsoft, Azure API Management policy reference - llm-emit-token-metric
• Microsoft, Plan and manage costs for Microsoft Foundry
• Microsoft, Allocate Azure costs