Installing the NVIDIA GPU Operator

Why it matters

An inference replica is only useful on a node whose driver, container runtime and device plugin all work. The GPU Operator is how NVIDIA provisions that stack on Kubernetes, so its install and health checks are where GPU scheduling starts.

What to practise

Install the gpu-operator Helm chart, then check that its Pods are Running or Completed, that kubectl get clusterpolicy reports ready, and that each GPU node advertises nvidia.com/gpu as allocatable. Run the CUDA VectorAdd Pod, which sets an nvidia.com/gpu: 1 limit. Then read the deployment scenarios: driver.enabled=false for nodes with a preinstalled driver, and toolkit.enabled=false as well when the NVIDIA Container Toolkit is preinstalled too.

How it fits

This is the provisioning layer of 4.4.1. Compare the operator’s validation with the health probes 4.4.1 adds: the operator proves a node can run GPU work, while probes decide when an inference Pod receives traffic.