Retriever API Usage with Multimodal Query Support

Why it matters

A question that arrives with a photo is either a question for a vision-language model or a retrieval query. This exercise makes the image part of the retrieval query itself.

What to practise

Stand up the NVIDIA RAG Blueprint with Docker Compose — a vector database, NIMs for the llama-nemotron-embed-vl-1b-v2 multimodal embedder and a vision-language model, the ingestor and the RAG server — or point them at NVIDIA-hosted endpoints. Ingest a PDF of product images and prices, then send a text-and-image query in the OpenAI vision message format, the image as a base64 data URL. Call /v1/search to see which documents come back and /v1/generate for a cited answer. Watch ENABLE_VLM_INFERENCE, vdb_top_k, enable_reranker and reranker_top_k.

How it fits

This is the retrieval branch of 2.2.1: compare it with sending the same image straight to the model through POST /v1/chat/completions, and decide which questions need the document collection at all.