NVIDIANVIDIA-Certified Professional: Agentic AI2026-107.2 Deploy NVIDIA NIM microservices for high-performance inference.
Choose how Kubernetes manages NIM deployments
Completed by the knowledge check
Learning objective
Deploy NIM on Kubernetes with Helm or the NIM Operator.
Core idea
Helm and the NIM Operator give you different application management interfaces. With the NIM chart, you manage a release through chart values. With the Operator, you declare NIM-specific resources, including NIMService and optional NIMCache, and its controller manages the corresponding workloads.
Both paths still require usable GPU nodes, compatible model profiles, artifact access, and storage. Installing a controller or submitting a release does not prove the model can serve requests. See 4.4.1 for GPU Pod admission and 7.2.1 for endpoint verification. Here the choice is which interface your platform team owns and how that interface connects the deployment’s dependencies.
How it works
Deploy a chart release
The NIM Helm page specifies Kubernetes 1.26 or later and Helm 3.0.0 or later, together with GPU-capable nodes and persistent storage for model caching. Check the cluster’s supported release separately: a documented minimum is not a recommendation to operate an unsupported Kubernetes release.
Fetch the versioned nim-llm chart from https://helm.ngc.nvidia.com/nim/charts/, using the chart version selected from the NGC Catalog. The documented download uses NGC credentials. After extracting it, inspect helm show readme nim-llm/ and helm show values nim-llm/; values are an interface owned by the chart version, not arbitrary container settings.
The essential values establish distinct deployment concerns:
image.repositoryandimage.tagselect the container and its version.imagePullSecretssupplies registry credentials when required.model.ngcAPISecretsupplies model-download credentials when required.persistenceconfigures cache storage, including its storage class, size, and access mode.resourcessets resource requirements, includinglimits.nvidia.com/gpuaccording to the selected profile.envsupplies runtime variables such asNIM_MODEL_PROFILE.customArgssupplies backend arguments as separate option and value list entries.
After preparing values.yaml, helm install my-nim nim-llm/ -f values.yaml creates the release. The deployment artifact worth reviewing is the combination of chart version and values, including its profile selector. Verify the resulting workload and endpoint.
Use customArgs rather than replacing NIM_PASSTHROUGH_ARGS with only your additions. The chart documentation warns that Kubernetes replaces an image-provided environment value rather than appending to it. customArgs preserves the image’s passthrough value and applies the added arguments at direct-argument priority. The flags still belong to the backend in the selected image; a vLLM argument cannot be assumed to work for SGLang.
Declare an Operator-managed service
On Kubernetes, NVIDIA documents the nvidia/k8s-nim-operator Helm chart; on OpenShift, it documents the web console’s Software Catalog (OperatorHub before OpenShift 4.20). The operator-sdk installation path is marked development-only. Installing the controller is a platform step; creating NIMService objects deploys the application.
The installation page requires the GPU Operator, cluster-admin access, and either an active NVIDIA AI Enterprise subscription or NVIDIA Developer Program membership. See 4.4.1 for GPU Operator responsibilities. Include those prerequisites when deciding who owns the Operator.
NIMService uses apiVersion: apps.nvidia.com/v1alpha1 and represents the resource nimservices.apps.nvidia.com. Its spec.image identifies repository, tag, pull policy, and pull secrets. spec.authSecret identifies the generic secret used for model-download credentials. The service reference marks that field required.
Use spec.resources.limits.nvidia.com/gpu for the GPU allocation and spec.replicas for a fixed replica count. spec.env carries environment variables, and spec.args carries container arguments for a single-node service. These are resource fields, not Helm values. For example, imagePullSecrets belongs to the chart interface, while the Operator examples use spec.image.pullSecrets.
The default Operator probes target /v1/health/live and /v1/health/ready. Customize them through spec.livenessProbe.probe and spec.readinessProbe.probe when the deployment needs a different probe specification. Preserve the difference between container liveness and loaded-model readiness established in 7.2.1.
Choose cached models or service-owned downloads
NIMCache, represented by nimcaches.apps.nvidia.com, manages model caching separately from the service. A NIMService attaches it through spec.storage.nimCache.name, with an optional spec.storage.nimCache.profile. Inspect the cache’s .status.profiles to identify the cached profile instead of inventing a name. Cache status Ready means the download and caching job succeeded; it is not the service’s inference readiness.
Without a separate cache resource, spec.storage.pvc lets the service download profiles into persistent storage. Its create, name, size, storageClass, and volumeAccessMode fields define that storage arrangement. If both storage paths are set, spec.storage.nimCache takes precedence over spec.storage.pvc.
The cache page documents an operational ownership consequence: deleting a NIMCache created with spec.storage.pvc.create: true also deletes the Operator-created persistent volume and claim. Establish who owns that object’s deletion before treating it as disposable deployment metadata.
When a cache reports NotReady or no status, inspect the cache Pod and events. NVIDIA documents large image downloads as one cause of prolonged ContainerCreating and temporary ReconcileFailed events. Confirm whether the image is still downloading before deleting and recreating the cache. A genuine Failed cache state instead means the job failed to download and cache the profile; investigate that job’s outcome.
Budget upgrade capacity and verify service
The chart defaults to a StatefulSet. Confirm its update strategy with helm show values nim-llm/ for the chosen chart version; statefulSet.enabled alone identifies the workload kind. A StatefulSet’s default .spec.updateStrategy.type: RollingUpdate deletes and recreates one Pod at a time, waiting for each replacement to become Running and Ready. It can reuse the released GPU without a surge replica.
Set statefulSet.enabled: false for Deployment mode; only then does the chart’s deployment.strategy apply. Compare the upgrade behaviors:
| Behavior | GPU headroom | Availability cost |
|---|---|---|
StatefulSet default RollingUpdate | Reuses each deleted Pod’s GPU | Five of six replicas serve during six sequential replacements |
Deployment RollingUpdate as described by NVIDIA | Room for a replacement before the old Pod terminates | Requires spare capacity for that surge |
Deployment Recreate | No surge GPU needed | All replicas stop before replacements start |
Budget sequential replacement time as well as spare GPUs. A delete-first rolling update avoids full interruption but can exceed the maintenance window when model loading is slow.
Inspect Pods, forward the Helm service with kubectl port-forward svc/my-nim-nim-llm 8000:8000, and check /v1/health/ready. Then discover the model and send a representative API request as in 7.2.1. For an Operator deployment, inspect the NIMCache and NIMService resources as well as their Pods.
Key distinctions and trade-offs
Choose the workload interface deliberately:
| Option | Deciding condition | Mechanism | Ownership and cost |
|---|---|---|---|
| NIM chart | Team already manages model releases through chart values | nim-llm, values.yaml, helm install | Team owns release values and storage configuration |
| NIM Operator | Platform standardizes NIM-specific service and cache resources | NIMService, NIMCache | GPU Operator, cluster-admin access, subscription or membership, controller and cache lifecycle |
| Separate cache | Model preparation needs an independently observable lifecycle | spec.storage.nimCache | Extra object and explicit deletion consequences |
| Direct persistent storage | Service owns profile download into its volume | spec.storage.pvc | Download readiness remains part of service preparation |
Helm also installs the Operator, so “uses Helm” alone does not identify who manages the serving workload. Read whether the proposed artifact is chart values or a NIM custom resource.
Helm appears on both paths, so whether chart values or a NIM custom resource follows it identifies who manages the serving workload.
Architecture / decision scenario
A legal-document research service has six approved one-GPU replicas and exactly six allocatable GPUs. Its two-person team manages Helm releases within a 10-minute maintenance window. Rehearsals measure 75 seconds from live to ready and 15 seconds for termination, scheduling, and reaching live: 90 seconds per replacement.
The platform owner confirms StatefulSet default rolling behavior in the selected chart values. Six sequential replacements take about 540 seconds, leaving five replicas serving during each replacement. That fits the 600-second window without a seventh GPU. The team keeps this mode instead of Deployment RollingUpdate, whose described surge cannot fit, or Deployment Recreate, which would interrupt all six replicas. If measured replacement time exceeded 100 seconds, the sequential estimate would exceed the window and require a revised window or an explicitly accepted interruption.
A proposal to standardize independently prepared caches is a separate reason to evaluate NIMCache and NIMService. The platform team must accept the GPU Operator prerequisite, cluster-admin installation, subscription or membership requirement, controller operations, and cache deletion ownership. Adding the Operator alone does not shorten the measured model-loading time.
Common exam traps
- Assuming Helm and Operator are mutually exclusive installation tools.
- Passing Helm value names as
NIMServicefields. - Confusing image-pull credentials with model-download credentials.
- Treating cache
Readyas proof of inference readiness. - Setting
deployment.strategywhile leaving StatefulSet mode enabled. - Deleting an Operator-created cache without accounting for its volume lifecycle.
What to remember
Choose chart values or NIM custom resources according to who operates the deployment. Configure image access, profile, GPUs, and persistent model storage through that interface, then verify cache preparation and actual inference separately. Upgrade policy must fit real GPU headroom.
Knowledge check
A team deploys NIM for LLMs with the nim-llm Helm chart and leaves its workload settings at the chart defaults. To avoid needing a spare GPU, the team adds deployment.strategy.type: Recreate to values.yaml. During the next upgrade, Pods are still replaced one at a time. What explains this?
Recreate into a delete-first rolling updatemaxSurge: 0 to replace the default strategydeployment.strategy applies only in Deployment mode, after statefulSet.enabled: false ; in StatefulSet mode the value is ignored. The StatefulSet’s default rolling update deletes and recreates one Pod at a time, which is the behavior observed. Setting statefulSet.enabled: false alongside the strategy makes Recreate take effect, with all replicas stopping before replacements start.A team moves a NIM deployment from the nim-llm chart to a NIM Operator NIMService. Its first manifest carries the chart’s imagePullSecrets and model.ngcAPISecret names. The images come from a private registry, and the model download needs NGC credentials. Where do the two credentials belong in the NIMService?
spec.env , keeping the chart’s value names exactly as they arespec , keeping the chart’s value namesspec.image.pullSecrets , which covers both credentialsspec.image.pullSecrets and spec.authSecret respectivelyspec.image.pullSecrets and name the model-download secret in spec.authSecret , which the service reference marks required. Image-pull and model-download credentials answer different questions — which registry may serve the container, and which source may serve the model — so one pull-secret list does not supply both.A NIM Operator deployment’s NIMCache reports Ready, and the NIMService Pods are in the Running phase. The team is about to send application traffic to the service. Which observation establishes that the service can take it?
.status.profiles entry for the selected profileReady status reported after the download jobRunning phase as reported by kubectl get pods for the serviceReady on a NIMCache means its job downloaded and cached the model profile, and .status.profiles says which profile it holds; neither says that the service loaded it. A running Pod can still be loading the model. Inference readiness comes from the service itself — /v1/health/ready , then model discovery and a representative request — so cache preparation and serving are verified separately.A team runs eight one-GPU NIM replicas with the nim-llm chart in its default StatefulSet mode, on exactly eight allocatable GPUs. Each replacement takes 100 seconds in rehearsal, the maintenance window is 12 minutes, and the service owner allows at most one replica out of service at a time. What should the team decide?
RollingUpdate , which needs no spare GPURecreate outageRollingUpdate as NVIDIA describes it needs room for a new Pod before the old one terminates, which eight fully used GPUs do not provide; without that surge, one Pod out at a time is again eight sequential replacements. Adopting the Operator does not shorten model loading. What remains is a longer window or an explicitly approved interruption, since Recreate stops every replica before replacements start.The nim-llm image supplies its own NIM_PASSTHROUGH_ARGS value. A team must add one vLLM backend argument to its chart release without losing that value. Which values.yaml change does this?
NIM_PASSTHROUGH_ARGS under env to the new argumentcustomArgs itemsNIM_MODEL_PROFILE value under envcustomArgs itemcustomArgs passes backend arguments directly to the container, preserving the image’s NIM_PASSTHROUGH_ARGS and applying the additions at the higher direct-argument priority; the chart asks for each option and each value as a separate list item. Setting the variable under env replaces the image’s value rather than appending to it, and a profile selector holds a profile ID or description, not backend flags.References
- NVIDIA, Helm and Kubernetes
- NVIDIA, Installing NVIDIA NIM Operator
- NVIDIA, Managing NIM Services
- NVIDIA, Caching NIM Models
- Kubernetes, StatefulSets: Rolling Updates