Skip to lesson

Google CloudProfessional Agentic Architect2026-094.2 Deploying and scaling production workloads

Balance production objectives

Completed by the knowledge check

Learning objective

Optimize a production agent across quality, performance, reliability, hallucination rate, and cost. This lesson focuses on choosing among measured production configurations, not on diagnosing individual failures or tuning one component in isolation.

Core idea

Treat production optimization as a constrained multi-objective problem. Do not search for a configuration that is “best” on every metric; such a configuration often does not exist. First set minimum acceptable outcomes, then choose the best business trade-off among configurations that meet them.

For example, accept a candidate only when all of these conditions hold:

quality >= Q_min
p95 end-to-end latency <= L_max
successful-task rate >= R_min
hallucination rate <= H_max
cost per successful task <= C_max

Among the acceptable candidates, prefer a Pareto-efficient candidate: no other candidate improves one objective without making at least one other objective worse. Business priority then selects a point on that frontier. A weighted score can rank acceptable candidates, but it must not average away a hard limit.

Google Cloud’s AI and ML performance guidance likewise recommends translating business goals into measurable objectives and distinguishes quality from performance. It warns that the highest-performing model might not produce the best business outcome when it requires more time and cost. See AI and ML perspective: Performance optimization.

How it works

  1. Segment the workload. Separate request classes that have different value, risk, or latency needs. An account-balance lookup and a regulated policy explanation should not inherit one undifferentiated target.

  2. Define each indicator precisely. Record the population, measurement window, aggregation, evaluator, and threshold. Useful indicators include:

    • Quality: task-completion rate or rubric pass rate on representative requests.
    • Performance: end-to-end latency percentiles, time to first token, and sustainable throughput at expected load.
    • Reliability: the fraction of eligible tasks that complete successfully, including model, orchestration, retrieval, and tool failures.
    • Hallucination rate: the fraction of evaluated responses or claims that lack required support. State the unit and evidence source. A grounding score and a hallucination rate are related, but they are not interchangeable without a defined conversion.
    • Cost: model, retrieval, tool, evaluator, runtime, and retry costs divided by successful tasks, not merely by requests.

    Google Cloud’s managed agent hallucination metric checks claims against tool usage evidence and reports a support-oriented score. Its final-response quality metric uses task-specific rubrics. These separate metrics show why a grounded response can still be unhelpful, incomplete, or instruction violating. See Details for managed rubric-based metrics.

  3. Measure a baseline and controlled candidates. Use the same representative evaluation set, load profile, tool versions, and traffic segments. Change one coherent design choice at a time, such as a routing policy or verification path. Record confidence intervals or sample counts so a small apparent gain does not become a production decision.

  4. Apply constraints before ranking. Reject a cheap candidate that exceeds the hallucination limit. Reject a high-quality candidate that misses the latency or cost limit. Compare business value only after these gates.

  5. Validate under production conditions. Roll out gradually, observe every objective together, and keep a rollback rule. Agent Platform observability exposes latency percentiles, errors, token use, response quality, and hallucination signals on related dashboards. Online monitors can sample production traces and cap samples per run to control evaluation cost. See Observability overview and Continuous evaluation with online monitors.

Key distinctions and trade-offs

Quality versus hallucination: Hallucination is one quality failure class, not the inverse of total quality. A fully supported answer can fail the user’s task, and a fluent answer can contain an unsupported claim. Track both.

Performance versus reliability: Performance describes speed and capacity; reliability describes whether the service produces the required result consistently. A short timeout can improve reported latency for successful calls while increasing failures. Include failed and timed-out work in end-to-end service indicators. Google’s reliability guidance recommends user-centered service-level objectives (SLOs) for success rate, latency, harmful responses, and agent task completion. See AI and ML perspective: Reliability.

Cost per request versus cost per success: A cheaper request is not cheaper when low quality causes retries, escalation, or rework. Use cost per successful task for the decision and retain component costs for diagnosis.

Common levers move several objectives at once. More reasoning or a more capable model can improve difficult-task quality but add latency and cost. Retrieval, tool calls, and verification can reduce unsupported answers, but each adds cost, latency, and a failure dependency. Model routing can reserve the expensive path for complex requests; Google Cloud’s agentic architecture guidance describes routing by task complexity, cost, and latency. Redundancy or reserved capacity can improve availability and throughput, but idle capacity raises cost. For Provisioned Throughput, unused throughput does not accumulate, so size it from prompt size, response size, and request rate rather than request count alone. See Calculate Provisioned Throughput requirements.

Architecture / decision scenario

A customer-support agent answers ordinary product questions and explains account-specific billing decisions. The team defines separate targets for each class. Billing explanations require a lower hallucination rate and higher task success; ordinary questions have a tighter latency and cost ceiling.

Testing shows that one powerful model with retrieval and verification on every request has the best average quality, but violates the latency and cost limits. A smaller model for every request meets those two limits but fails quality and hallucination thresholds on billing explanations.

The team chooses a routed design. A smaller model handles low-risk questions. Billing requests use authoritative retrieval, a more capable model, and a claim verification step. If required evidence is absent, the agent abstains and escalates rather than guessing. The team load-tests both paths, deploys the router as a canary, and monitors cost per successful task alongside p95 latency, task success, failures, and hallucination rate for each segment.

This design does not maximize one global score. It meets every constraint and spends extra latency and cost only where the business risk justifies them.

Common exam traps

  • Choosing the configuration with the highest quality score even though it violates a reliability, latency, hallucination, or cost constraint.
  • Combining all metrics into one average before enforcing hard thresholds.
  • Treating low hallucination as proof of correctness, usefulness, safety, or successful task completion.
  • Measuring only average latency and hiding slow tail requests or timeouts.
  • Calling a successful HTTP response reliable when the agent failed the user’s task.
  • Optimizing token price per request while ignoring retries, tool calls, evaluation, escalation, and failed work.
  • Using one target for request classes with different risk and value.
  • Trusting offline evaluation alone or changing several variables during a production experiment without preserving attribution.

What to remember

  • Convert business priorities into explicit indicators, thresholds, and request segments.
  • Enforce hard constraints before ranking acceptable configurations.
  • Track quality and hallucination separately, and distinguish performance from reliability.
  • Optimize cost per successful task rather than cost per request.
  • Choose a Pareto-efficient design, roll it out gradually, and monitor every objective together.

Knowledge check

References