Skip to lesson

NVIDIANVIDIA-Certified Professional: Agentic AI2026-100 Prerequisites

Separate images, workloads, and service addresses

Completed by the knowledge check

Learning objective

Explain Docker images and Kubernetes pods, deployments, and services.

Core idea

An image packages software; a container runs it. Kubernetes places containers in Pods, controllers maintain the desired workload, and Services expose network endpoints. These objects answer different questions: what runs, where it runs, how it is replaced, and how callers reach it.

For an inference application, distinguish the packaged model server from the agent application that calls it. Sharing a container platform does not make them the same workload or give them identical scaling needs.

How it works

A Docker image is a read-only template used to create containers. A Dockerfile describes its construction. A registry stores images for distribution. A container is a runnable image instance plus runtime configuration; removing it discards changes that are not in persistent storage. Packaging a server is therefore different from preserving its runtime data.

A Pod is Kubernetes’ smallest deployable unit. It holds one or more co-located, co-scheduled containers with shared networking and storage resources. Containers that must run together can share a Pod. Independent agent and inference services usually need separate workload lifecycles rather than being grouped merely because they communicate.

A Deployment describes a desired set of Pods. Its .spec.template defines the Pod template, .spec.replicas sets the desired count, and .spec.selector identifies the managed Pods. A ReplicaSet maintains that count; changing the template starts a controlled rollout. Pods are disposable instances, so depend on the controller’s desired state rather than treating an individual Pod as a permanent server.

A Service exposes a logical set of endpoints. With a selector-based Service, .spec.selector matches Pod labels and Kubernetes maintains the corresponding EndpointSlices. .spec.ports describes the exposed ports. A ClusterIP Service provides an internal cluster address; callers need not track each Pod’s changing IP. Creating that Service does not create the application containers.

Key distinctions and trade-offs

A Deployment controls workload lifecycle; a Service controls network access to endpoints. Scaling .spec.replicas changes the desired number of Pods, whereas changing a Service selector changes which Pods its traffic reaches. Neither action changes the model’s numerical precision.

The two objects reach the same Pods from different sides: the Deployment sets the desired number of Pods, and the Service decides which Pods callers’ traffic reaches.

A Deployment sets the desired number of Pods and a Service decides which Pods callers' traffic reachesOn the workload lifecycle side, a Deployment sets the desired count and a ReplicaSet maintains that count of Pods. On the network access side, callers use the ClusterIP Service's internal cluster address, and its selector matches Pod labels. Both sides end at the same Pods, so callers need not track each Pod's changing IP.network accessworkload lifecycledesired countinternal cluster addressmaintains that countselector matches Pod labelscallersClusterIP ServiceDeploymentReplicaSetPods

Readiness is also different from process existence. NIM for LLM and VLM documents /v1/health/live for container liveness and /v1/health/ready for a loaded model available for inference. A running container can still be loading its model. See 4.4.1 for GPU resource requests and health probes, and 7.2.3 for NIM deployment through Helm or the NIM Operator.

Architecture / decision scenario

A manufacturer runs an on-premises maintenance assistant with six agent replicas and two approved inference replicas. The platform owner must update the agent prompt-processing code without restarting model serving, whose measured startup takes four minutes in this example.

One proposal puts an agent and model server together in every Pod. That ties the rollout and replica counts together, creating six model instances where the GPU budget permits two. The owner instead uses separate Deployments, with .spec.replicas set to six and two, and an internal Service selecting inference Pods by label. Agent Pods call the Service address.

During a model rollout, the readiness check must represent model availability rather than merely container liveness. When a configured readiness probe fails, Kubernetes removes the Pod from matching Services’ endpoints. Pointing the probe at /v1/health/ready rather than /v1/health/live keeps ordinary Service traffic off a replacement still loading its model. Detailed probe configuration belongs to 4.4.1; the prerequisite decision is which object owns each responsibility.

Common exam traps

  • Treating an image as a running process or a durable data store.
  • Treating a Pod IP as a permanent application endpoint.
  • Expecting a Service to maintain the replica count.
  • Equating a live model-server container with inference readiness.

What to remember

Build an image, run containers in Pods, maintain suitable workloads with a Deployment, and expose endpoints through a Service. Keep lifecycle, network identity, readiness, and capacity as separate decisions.

Knowledge check

References