NVIDIANVIDIA-Certified Professional: Agentic AI2026-107.2 Deploy NVIDIA NIM microservices for high-performance inference.
Match serving profiles to available GPUs
Completed by the knowledge check
Learning objective
Select NIM model profiles and inference backends for the target GPU.
Core idea
A NIM profile is a prevalidated inference configuration, selected at container startup. It combines backend, precision, and parallelism choices; choosing a model name alone leaves those decisions unresolved. The selected profile determines the model files to download and how the backend launches.
Select from the profiles the intended image actually provides on the target hardware. list-model-profiles exposes those choices, and NIM_MODEL_PROFILE selects an ID or complete description. See 3.4.2 for quality versus precision and 4.5.1 for demand sizing. Here the decision is whether a particular deployment configuration can run, and whether its effective runtime settings match what you approved.
How it works
Inspect the manifest on the target system
A model-specific NIM container includes a model manifest containing one or more profiles. A model-free NIM generates its manifest at runtime with generic profiles. At startup, NIM selects exactly one profile from that manifest.
Before procuring or allocating GPUs, read the Support Matrix row for your model and candidate GPU. Check the supported precision, tensor parallelism, and profile ID; optimized-profile tables distinguish configurations that share precision and parallelism, and descriptions can carry a GPU-name tag. Then confirm the planned configuration on the target system with list-model-profiles.
Run list-model-profiles inside the intended image with access to the target GPUs. The interface is:
docker run --rm --gpus=all "$NIM_LLM_IMAGE" list-model-profilesSet NIM_LLM_IMAGE to the chosen supported image and tag. The output includes profile IDs, descriptions, and estimated per-GPU memory requirements. Keep that output with your deployment configuration; a description guessed from a naming convention can omit a GPU or performance tag that distinguishes two configurations.
The naming convention defines backend tags vllm, sglang, and trtllm; the 2.0.13 Support Matrix lists vLLM and SGLang profiles rather than trtllm profiles. Other tags encode precision (bf16, fp8, mxfp4, or nvfp4) and tensor parallelism (tp). The documented description vllm-bf16-tp4-pp1-lora describes BF16 across four GPUs with one pipeline stage and LoRA support. It explains the encoding, not availability for every model or GPU. Copy the entire description from your listing, or use its 64-character profile ID.
The backend must match the engine in the image. An SGLang image loads sglang-* profiles; setting a vllm-* selector does not turn that image into vLLM. Compare backend candidates by listing their respective images, then compare the configurations they actually expose. Do not prescribe a universal engine winner from the names alone.
Interpret compatibility and memory separately
The selection page distinguishes three memory outcomes. A compatible profile’s estimated memory fits. A low-memory profile has room for model weights but not the full context length. An incompatible profile’s weights alone exceed available memory. The memory estimate considers weights, key-value cache, activations, and overhead.
For low memory, the listing can suggest a reduced --max-model-len. NVIDIA’s illustrative output shows a profile estimated to require at least 45 GB per GPU at its full context, reduced to at least 30 GB per GPU with --max-model-len=4096. These are that documentation example’s estimates, not a named model’s support guarantee. The useful mechanism is the listing’s concrete suggestion on your machine.
A lower context limit changes the agent’s usable input budget. Before accepting it, check whether retrieval passages, conversation history, and the requested output still fit the application’s workload. A profile that starts only after dropping necessary context has solved initialization while breaking the task.
If weights alone do not fit, the documented remedies are higher tensor parallelism or a quantized profile. More tensor parallelism requires more GPUs per replica; quantization requires checking task quality. A smaller context limit addresses the low-memory case, not weights that already exceed memory. This distinction prevents repeated restarts with an irrelevant context flag.
Choose automatic discovery or explicit selection
Leaving NIM_MODEL_PROFILE unset lets NIM select a compatible manifest profile based on GPU device, available memory, estimated requirements, and parallelism constraints. Setting it to "default" invokes the documented intelligent default selection, which considers hardware compatibility, backend priority, and LoRA configuration. Neither expresses your application’s measured latency or quality target.
For a controlled rollout, set NIM_MODEL_PROFILE to the exact profile ID or full description from the listing. Explicit selection makes the intended profile reviewable. It does not reserve GPU capacity, add missing memory, or make an incompatible backend image usable.
Treat the container image tag and profile selector as one deployment choice. A profile is drawn from that image’s manifest, so preserving only a human-readable model name is insufficient for reproducing the selection. During an upgrade, inspect the candidate image’s listing again rather than assuming the previous image’s catalog remains unchanged.
Check the effective runtime configuration
A selected profile is not the final authority over every backend setting. Direct container arguments override NIM_PASSTHROUGH_ARGS, which overrides NIM environment variables mapping to backend arguments, which overrides runtime_config.json; these sources can override profile settings. NVIDIA illustrates a profile with TP=2 launched with the vLLM argument --tensor-parallel-size 4, yielding TP=4.
Consequently, review arguments as well as NIM_MODEL_PROFILE. A deployment can claim a two-GPU profile while an inherited argument requests four GPUs. Remove an unintended override or approve the changed GPU allocation explicitly. Backend flag names differ: a vLLM option is not automatically an SGLang option.
NIM’s profile page also identifies removed 1.x selectors: NIM_CUSTOM_SELECTOR_CLASSES, NIM_TAGS_SELECTOR, and the 1.x automatic TensorRT-LLM-to-vLLM-to-SGLang priority chain. Use the supported profile interface rather than copying those selectors into a deployment. A remembered backend ordering is not a substitute for the chosen image’s catalog and documented selection behavior.
Key distinctions and trade-offs
Choose the mode that matches the decision you need to control:
| Choice | Deciding condition | Mechanism | Cost or limitation |
|---|---|---|---|
| Automatic selection | Discover a compatible startup configuration | Leave NIM_MODEL_PROFILE unset | Compatibility does not prove workload performance |
| Intelligent default | Request the documented default selector | NIM_MODEL_PROFILE="default" | Does not express application quality or latency targets |
| Explicit selection | Reproduce an approved configuration | Full profile ID or description | Must match the image and target hardware |
| Reduced context | Weights fit; full context does not | Listing’s suggested --max-model-len | Less room for the agent’s request |
| Higher TP or quantization | Weights do not fit | Suitable listed profile | More GPUs or a precision-quality trade-off |
These choices operate within backend compatibility. A profile selector chooses from the image’s supported configurations; it does not supply a missing inference engine.
Architecture / decision scenario
An industrial research assistant has a fixed allocation of two GPUs and requires 12,000-token requests. In a deployment rehearsal, its candidate listing classifies the full-context configuration as low memory and suggests a 4,096-token maximum. Another candidate profile requires four GPUs.
A practitioner proposes accepting the context reduction because the server then becomes ready. The inference owner rejects it: 4,096 cannot satisfy the required 12,000-token workload, and four GPUs exceed the allocation by two. The owner instead evaluates a listed quantized candidate that fits the two-GPU allocation, with the application team checking research-answer quality as described in 3.4.2.
The team lists profiles separately for its vLLM and SGLang images, chooses the successful image-profile pair, and records the exact selector. Before rollout, it removes an inherited --tensor-parallel-size 4 argument from the vLLM candidate. Readiness and representative inference then validate the effective two-GPU deployment. This chooses among context reduction, more parallelism, and quantization without mistaking any one of them for free capacity.
Common exam traps
- Constructing a selector from a shortened description instead of copying the listed value.
- Assuming
NIM_MODEL_PROFILEcan switch the backend embedded in an image. - Treating low memory and weights-too-large as the same condition.
- Calling a context reduction successful when required requests exceed it.
- Ignoring direct arguments that override the selected profile’s settings.
- Reusing removed 1.x selector mechanisms as deployment advice.
Where the exam guide differs
The September 2025 study guide’s “Deploy NVIDIA NIM microservices for high-performance inference” subsection names no NIM version. NIM LLM 1.x selected backends automatically in TensorRT-LLM > vLLM > SGLang priority order. The 2.0 migration guide describes a vLLM-based architecture; the 2.0.13 Support Matrix also documents SGLang images for some models. Select profiles with NIM_MODEL_PROFILE from the chosen image’s own manifest.
Read an exam answer describing a GPU-matched optimized profile selected by the platform, with backend fallback, as the 1.x model. For a 2.0 deployment, check the model’s documented image and profiles rather than applying that removed backend-selection chain.
What to remember
List profiles on the intended hardware, choose an image-compatible selector, and inspect memory classification before changing context or parallelism. Preserve the image-profile pair and review runtime overrides so the effective GPU configuration matches the approved one.
Knowledge check
A research assistant’s NIM for LLMs replica has a fixed allocation of two GPUs and must accept 16,000-token requests. On the target system, list-model-profiles places the BF16 TP2 profile under “Incompatible with system”. What should the inference owner evaluate next?
--max-model-lenNIM_MODEL_PROFILE to the BF16 TP2 profile’s full IDA team runs the SGLang image of a model and wants to know whether the vLLM image would serve it better. An engineer proposes setting NIM_MODEL_PROFILE on the SGLang deployment to a vllm-* description copied from a colleague’s listing. Which approach gives a valid comparison?
vllm-* description on the SGLang deploymentlist-model-profiles in each image and compare their profilesNIM_MODEL_PROFILE="default" to choose by backend prioritysglang-* profiles — so a selector chooses within one image’s manifest and cannot switch its engine. Listing each image on the target hardware shows which configurations each one actually exposes for a like-for-like test. The default selector also picks only from the running image’s profiles, and vLLM flag names are not SGLang options.A vLLM-based NIM for LLMs deployment selects a TP2 profile through NIM_MODEL_PROFILE using its full ID. The pod template inherits NIM_PASSTHROUGH_ARGS="--tensor-parallel-size 4" and passes no direct container arguments; the node exposes four GPUs, and the image’s listing also contains a TP4 profile of the same precision. Which statement describes the launch?
NIM_PASSTHROUGH_ARGS , then NIM environment variables mapped to backend arguments, then runtime_config.json . The effective TP setting is therefore 4 even though the selector names a TP2 profile. A reviewer must remove the inherited argument or approve the larger allocation explicitly; --tensor-parallel-size is the vLLM flag.A team writes NIM_MODEL_PROFILE from the naming convention as a shortened description, vllm-fp8-tp2. On the target GPUs, the listing shows a longer description for that configuration that also carries a GPU tag and a performance tag. The change review must name exactly which profile will run. How should the team set the selector?
NIM_MODEL_PROFILE="default" and record only the image tagReferences
- NVIDIA, Model Profiles and Selection
- NVIDIA, Environment Variables
- NVIDIA, Quickstart
- NVIDIA, NIM LLM 1.x to NIM LLM and VLM 2.0 Migration Guide
- NVIDIA, Support Matrix for NIMs