NIM for LLMs Quickstart

Why it matters

Every self-hosted model in an agent starts as one container that has to come up, load weights and answer correctly. This quickstart is that path at its smallest.

What to practise

Start a model-specific NIM with docker run --gpus=all, a mounted LOCAL_NIM_CACHE, and NGC_API_KEY only when the image needs it — or a model-free NIM with NIM_MODEL_PATH and HF_TOKEN. Query /v1/models for the served name, call /v1/health/live and /v1/health/ready, then send a request to /v1/chat/completions, with and without streaming. Watch how long the gap between live and ready lasts on first start, and how a warm cache changes it.

How it fits

The health-then-completion sequence is 7.2.1’s validation path. The automatic profile choice the quickstart relies on, and its NIM_MODEL_PROFILE override, are 7.2.2 — decide when you would pin a profile instead of trusting detection.