Choose models by their required capabilities

Learning objective

Distinguish foundation model families and capabilities (chat, reasoning, small, multimodal, and embedding models)

Core idea

Choose a model by the operation your system needs, then verify its supported interface. Chat, reasoning, small, multimodal, and embedding describe different properties. They aren’t five mutually exclusive product categories. A small model can accept conversation messages; a reasoning model can accept images; an embedding model can serve retrieval without generating an answer.

For an architect, the useful question is whether the deployed model can perform the required transformation within the workload’s quality, latency, and resource constraints. A recognizable family name doesn’t establish tool support, output format, or deployment availability. Microsoft Foundry documents those properties by model and version. See 0.2 for token budgets and inference settings; this lesson establishes which kind of model belongs at each stage.

How it works

Separate the capability dimensions

Use the following dimensions to describe a candidate before comparing it with another model:

These dimensions help prevent an invalid substitution. Replacing a chat deployment with an embedding deployment doesn’t give you a cheaper answer generator. Adding image input doesn’t make a text-output model an image generator. Moving to a reasoning model also requires checking request parameters rather than preserving every sampling setting from the previous adapter.

Use chat for generated responses

The Azure OpenAI Chat Completions API takes messages, an array of role-bearing message objects. A system message supplies application instructions, a user message supplies the request, and assistant messages represent conversation responses or examples. This interface also supports nonconversational tasks such as classification and summarization. A chat interface doesn’t require a chat user experience.

The Foundry model catalog lists gpt-4.1, gpt-4.1-mini, and gpt-4.1-nano with text and image input and text output. It lists function calling and structured outputs with their supported API scope. Treat those entries as a capability checklist, not evidence that every chat model supports the same features. In Azure requests, model identifies your deployment name; it isn’t necessarily the catalog’s model ID.

Use reasoning where deliberation earns its cost

Azure OpenAI reasoning models generate reasoning tokens in addition to visible output. Those tokens consume context and are billed as output tokens. They don’t appear in the message content. The benefit must therefore be measured against both task quality and the computation spent to obtain it.

A reasoning model is a candidate for a task with interacting constraints, planning, or difficult comparisons. It doesn’t make authoritative facts available by itself. For a reconciliation task, provide the actual records and validate the resulting decision. Raising reasoning_effort cannot repair missing evidence. Supported effort values and tool behavior depend on the model and API; see 0.2 for those controls.

Evaluate small models against a bounded task

Small models prioritize a different resource and capability balance from larger candidates. Consider them for a narrowly specified extraction, classification, or transformation task with an explicit acceptance test. Don’t assume a smaller candidate is suitable merely because the output is short: deciding the correct label can require substantial domain reasoning.

Microsoft’s catalog illustrates why capabilities matter more than a size label. It lists Phi-4-mini-instruct as text input and text output and marks tool calling as unsupported. It separately lists Phi-4-mini-reasoning with reasoning content. If your application needs documented function calling, that catalog entry rules out the mini-instruct deployment for that interface, even if it performs well on text classification.

A model’s compactness also doesn’t prove that its total application cost is lower. Compare accepted results, retries, and escalation frequency. An inexpensive first pass that repeatedly needs repair can cost more per successful task than a stronger candidate.

Specify both sides of multimodality

Multimodal means support for more than one modality, not unrestricted media conversion. The catalog lists Phi-4-multimodal-instruct with text, image, and audio input but text output. A workflow that summarizes a spoken report can fit that direction. For speech output, the catalog names the /audio API for speech to text, translation, and text to speech. Its text-to-speech subsection is labeled preview and includes gpt-4o-mini-tts; tts and tts-hd are also marked Preview. Select that output capability explicitly rather than inferring it from audio input.

Likewise, the GPT-4.1 catalog entries accept images but output text. For a photographed invoice, verify image-input support and evaluate whether the extracted fields match the document. For generating an illustration, the catalog names image-generation models such as gpt-image-1, gpt-image-1-mini, gpt-image-1.5, and gpt-image-2, which create images from text prompts. Keep separate adapters when payload types differ; don’t send a text-only request shape and assume media support activates itself.

Use embeddings for representation and retrieval

An embedding is a vector of floating-point numbers representing semantic meaning. Similar text produces nearby vectors, which supports vector search, recommendations, classification, and clustering. Azure OpenAI exposes this operation through client.embeddings.create; the generated vector is returned in data[0].embedding.

The catalog gives text-embedding-3-small an 8,192-token input limit and 1,536 output dimensions, and text-embedding-3-large the same input limit and 3,072 dimensions. Third-generation embedding models support the dimensions parameter to reduce vector size. Vector size affects storage and comparison work; it isn’t the length of an answer.

Changing embedding models requires regenerating embeddings. Preserve a consistent model and dimension choice for the document and query representation you compare. An embedding score locates candidates; it doesn’t establish that a retrieved claim is correct.

Key distinctions and trade-offs

Interface versus intelligence: Chat describes how you exchange messages. Reasoning describes additional computation. Neither label establishes all supported tools or modalities.

Model size versus task fit: A small model can meet a narrow target, but size alone doesn’t establish quality or end-to-end savings. Evaluate the candidate against representative inputs and failure cases.

Representation versus generation: An embedding supports similarity operations. A generative model consumes evidence and produces an answer. A retrieval-assisted system can need both.

Architecture / decision scenario

An insurance support system handles typed questions, photographed claim forms, routine routing, and disputes with several interacting policy clauses. Use embeddings to locate relevant policy passages. Use a chat-capable deployment to explain those passages in text, with an image-capable input path for the photographed form.

Evaluate a compact candidate for routine routing using labeled examples and a quality threshold. Reject a deployment whose documented interface lacks a required tool capability. Route difficult policy comparisons to an evaluated reasoning candidate with the actual clauses in context. Keep factual validation after generation rather than treating reasoning as proof.

If the product also needs spoken answers, image input or audio input alone doesn’t satisfy that requirement. Select an audio-output component explicitly, such as a text-to-speech model through /audio, with its documented preview status. Record model version, API, input modality, output modality, and required capabilities for every stage so a substitution can be checked before deployment.

Each request kind reaches a different model kind, and both generated answers pass through factual validation.

Common exam traps

• Treating chat, reasoning, small, and multimodal as mutually exclusive families.
• Expecting an embedding model to generate a conversational answer.
• Assuming image or audio input implies the same output modality.
• Assuming every compact chat model supports function calling.
• Comparing token prices without counting retries and escalations.
• Changing an embedding model while keeping incompatible stored vectors.

What to remember

Describe the required transformation first. Verify the deployed model’s API, modality direction, tool support, and output contract. Use embeddings for representation, chat for generated responses, and reasoning when evaluated deliberation improves the result enough to justify its cost. Treat small and multimodal as additional dimensions of that choice.

Knowledge check

References

• Microsoft, Foundry Models sold directly by Azure
• Microsoft, Foundry Models from partners and community
• Microsoft, Azure OpenAI reasoning models
• Microsoft, Use the Chat Completions API
• Microsoft, Generate embeddings with Azure OpenAI