Build and Deploy a Multi-Agent Chatbot

Why it matters

Several models serving several agents on one machine makes every service boundary and memory budget visible. It is a compact rehearsal for a multi-agent deployment.

What to practise

On a DGX Spark, download the model files and bring the stack up with docker compose -f docker-compose.yml -f docker-compose-models.yml up -d --build. A supervisor agent on gpt-oss-120B orchestrates downstream agents for coding, retrieval-augmented generation and image understanding, using MCP servers as tools, with models served by llama.cpp and TensorRT LLM servers. Open the UI on port 3000, which talks to the backend on port 8000, upload a PDF to try the RAG agent, and watch the default setup use about 120 GB of the 128 GB of memory — the playbook’s fallback is a smaller gpt-oss-20B supervisor.

How it fits

The supervisor, the model servers, the backend and the frontend running as separate containers are 4.1.1’s split by execution and ownership. One Compose file on one host is the operating model 4.1.2 weighs against Helm.