Discover and evaluate models with Model Garden

Learning objective

Use Model Garden to discover candidate models and evaluate them against an agent’s capability, quality, performance, cost, security, and operational requirements. This lesson focuses on a selection workflow, not on model-size or model-sourcing theory.

Core idea

Treat model selection as a funnel: discover, qualify, then measure.

Model Garden is a catalog and integration surface, not proof of architectural fit. Use metadata to find candidates, documentation to reject candidates that violate hard constraints, and workload-specific tests to compare the survivors.

Model Garden catalog -> architecture gates -> workload evaluation -> decision

Google Cloud describes Model Garden as a library for discovering, testing, customizing, and deploying models from Google and its partners. It groups models in one place and integrates with tuning, evaluation, and serving. See the Model Garden overview.

How it works

1. Turn architecture needs into a scorecard. Separate requirements that a model must satisfy from qualities that you can optimize. Hard gates can include input and output modalities, required context size, function calling, structured output, an approved serving location, a private network path, and acceptable license terms. Measured targets can include task success, p95 latency, throughput, and cost per successful task. Define each threshold before comparing models.

2. Use Model Garden to build a shortlist. Search the catalog, then filter by task, model collection, provider, and required features. Model Garden includes foundation models, fine-tunable models, and task-specific solutions; these are different starting points, not quality rankings. The available filters and categories are documented in Explore models.

3. Qualify each candidate from its model card and linked documentation. Record the exact model and version. Verify modalities, context limits, tool-use features, locations, quotas, pricing, license, and lifecycle. Also verify the serving path. Some models have a managed API and need no deployment; others require an endpoint and dedicated compute. Open models can sometimes be available through both a managed API and self-deployment. See Open model serving options.

4. Run a smoke test. For supported models, Model Garden can open prompt design so that you can submit prompts and adjust model parameters. This check confirms that the model accepts the expected input and produces a usable response shape. It does not establish production quality, latency, or cost. The available actions vary by model, as shown in Use models in Model Garden.

5. Evaluate the survivors on the same workload. Use representative normal, difficult, and failure cases. Keep the prompt template, tools, generation settings, and scoring rules fixed. Score the behavior the architecture needs, such as valid tool arguments, grounded answers, instruction following, or end-to-end task completion. The Gen AI evaluation service supports rubric-based metrics for qualities that are difficult to compute and computation-based metrics when a reference answer supports a deterministic comparison. See the Gen AI evaluation service overview.

6. Test the complete serving configuration. Measure tail latency, throughput, quota behavior, and effective cost with realistic context sizes, output lengths, concurrency, and retries. For a self-deployed model, include its machine type, accelerator, replica policy, and idle capacity. For a managed API, include its region, quota, consumption option, and request pricing. A model passes only when the exact model, version, configuration, and serving path meet every hard gate and all minimum targets.

Key distinctions and trade-offs

Availability is part of capability. A model that supports function calling on one endpoint type does not necessarily support it in every location or consumption option. Google Cloud publishes endpoint-specific capability and location constraints in Supported capabilities. Verify the combination that the architecture will use.

Governance is also separate from evaluation. A Model Garden organization policy can restrict projects to approved models, but inclusion on an allowlist does not prove that a model meets an application’s quality targets. See Control access to Model Garden models.

Architecture / decision scenario

A claims-intake agent must read text and document images, produce schema-valid claim data, and call a policy tool with correct arguments. The solution must run in an approved region, reach at least 97% valid tool calls on a curated test set, keep p95 response latency below two seconds at peak load, and remain within a fixed cost per completed claim.

The architect filters Model Garden for models that support the required task and features. Model cards produce a shortlist of three exact model versions. One candidate is removed because the required feature is not supported in the approved region. A smoke test confirms that the remaining two accept the multimodal request and tool schema.

Both candidates then receive the same representative claim cases, prompt, tools, and generation settings. Candidate A has the highest average answer quality but misses the tool-call threshold. Candidate B clears the quality gate. A production-like load test also shows that candidate B meets the p95 latency and cost limits on its selected serving path. The architect chooses candidate B and stores the evaluation inputs, configuration, results, and model version as the baseline for later upgrades. The highest-scoring candidate does not win; the candidate that satisfies the complete architecture contract does.

Common exam traps

• Selecting the model with the best public benchmark without testing the agent’s actual tasks and tools.
• Treating a Model Garden listing or model card as proof of workload quality.
• Treating one successful playground prompt as a production evaluation.
• Comparing candidates with different prompts, parameters, tools, or test cases.
• Ignoring the exact model version, region, endpoint type, or serving configuration.
• Averaging away a failed hard constraint, such as data location or required function calling.
• Assuming an organization allowlist measures model quality rather than model access.

What to remember

• Use Model Garden to discover candidates, not to skip evaluation.
• Apply hard architecture gates before ranking softer trade-offs.
• Verify the exact model, version, features, location, and serving path.
• Use prompt design for smoke tests and a repeatable dataset for comparison.
• Select the candidate that meets the full architecture contract, then preserve its configuration and results as a regression baseline.

Knowledge check

References

• Google Cloud, Overview of Model Garden
• Google Cloud, Use models in Model Garden
• Google Cloud, Open model serving options
• Google Cloud, Gen AI evaluation service overview
• Google Cloud, Supported capabilities
• Google Cloud, Control access to Model Garden models