Design the RAG lifecycle

Learning objective

Design a retrieval-augmented generation (RAG) lifecycle from source ingestion and chunking through retrieval and grounded generation. This lesson focuses on the end-to-end flow, not on choosing embedding models, tuning retrieval parameters, reranking results, or selecting a Google Cloud retrieval product.

Core idea

Treat RAG as two connected flows:

• An ingestion flow turns approved source content into searchable chunks.
• A serving flow finds relevant chunks for a request and gives them to the model as evidence.

The indexed chunk is the contract between the flows. It should contain a useful unit of content and enough metadata to identify, filter, authorize, refresh, and cite its source. Google Cloud describes the RAG pipeline as ingestion, transformation, embedding, indexing, retrieval, and generation in its RAG Engine overview.

How it works

The two flows, and the index where they meet:

1. Ingest approved sources. Connect to authoritative documents, databases, or content repositories. Preserve source identifiers, versions, timestamps, ownership, and access-control attributes. Ingestion copies or synchronizes knowledge; it does not answer a user query.

2. Parse and chunk the content. Extract usable text and structure, remove irrelevant formatting, and divide large documents into retrievable units. Keep headings, table context, source links, and authorization metadata with each chunk. Chunking creates the evidence unit that retrieval can return; tokenization creates the units a model processes. They are not the same operation. Google Cloud’s documented unstructured-data workflow uses a layout parser to create structured, context-aware chunks before embedding and indexing them in Vertex AI APIs for search and RAG.

3. Embed and index the chunks. An embedding represents a chunk numerically so that semantically related content can be found. Store the embedding with the chunk text and metadata in a searchable index. The embedding supports search; it is not the source evidence shown to the model or user.

4. Retrieve evidence for the request. At serving time, represent or rewrite the user’s query as required by the retrieval system. Apply identity-based permissions and request filters, then search the index for relevant chunks. Retrieval returns candidate evidence. It does not generate the final answer. For example, Agent Search can use an end user’s identity to restrict results to documents that the user can access, as described in data source access control.

5. Augment the model input. Construct a prompt that contains the original request, retrieved chunks, source metadata, and instructions about how to use the evidence. Stay within the model’s context limit and keep enough source context to avoid changing the meaning of a passage.

6. Generate and validate the grounded response. Ask the model to answer from the supplied evidence, attach source attributions, and decline or escalate when evidence is absent or conflicting. Grounding connects output to verifiable information; it reduces unsupported generation but does not make every response correct. A separate grounding check can compare claims with supplied facts and return support scores and citations, as described in Check grounding with RAG.

When source content changes, repeat the ingestion flow so that the serving flow does not rely on stale chunks. Record failures such as empty retrieval, outdated evidence, and unsupported claims so that you can improve the appropriate stage.

Key distinctions and trade-offs

RAG versus model tuning: RAG retrieves knowledge at request time and adds it to the prompt. Tuning trains on examples and adjusts model behavior or parameters. Use RAG for changing, private, or attributable knowledge; use tuning when the goal is consistent task behavior or output style. Google Cloud treats grounding and model tuning as separate adaptation methods.

Retrieval versus grounding: Retrieval selects evidence. Grounding is the relationship between generated claims and that evidence. Relevant chunks can still lead to an unsupported answer if prompt construction or generation fails.

Chunk text versus metadata: Text carries the evidence. Metadata carries its provenance and policy context. Discarding metadata can make accurate filtering, refresh, authorization, and attribution impossible.

The main lifecycle trade-off is context versus focus. Larger chunks can preserve surrounding meaning, but they consume more prompt capacity and can hide details. Smaller chunks can make embeddings more precise, but they can omit qualifiers or relationships. Google Cloud describes this chunk-size trade-off in Fine-tune RAG transformations. Later design steps tune chunking and retrieval against measured workload quality.

Architecture / decision scenario

A company builds an assistant for regional HR policy. The authoritative corpus contains global policies and country-specific addenda.

During ingestion, the team parses documents by heading and keeps each rule with its exceptions. Every chunk carries the policy version, effective date, country, source URL, and permitted employee group. The team embeds and indexes those chunks.

When an employee asks about parental leave, the serving flow applies the employee’s access group and country before retrieval. It sends the best permitted policy chunks, their titles, and their versions to the model with an instruction to answer only from those sources. The response cites the applicable policy. If retrieval finds no current approved policy, the assistant says that it lacks sufficient evidence and routes the employee to HR.

This design prevents a common failure: retrieving a semantically similar policy from the wrong country and presenting it as company-wide guidance. The critical decision is not only whether the content is similar, but whether it is authorized, current, attributable, and applicable.

Common exam traps

• Treating document upload as a complete RAG pipeline. Content must still be transformed, chunked, indexed, retrieved, and supplied to generation.
• Confusing chunks with embeddings. A chunk contains evidence; an embedding is a numerical representation used to find it.
• Assuming RAG changes model parameters. RAG augments the request-time context; tuning changes model behavior through training.
• Calling an answer grounded merely because retrieval occurred. Generated claims must remain supported by the retrieved evidence.
• Applying authorization after retrieval or generation. Filter the searchable evidence before it enters the model context.
• Returning an answer when retrieval provides no adequate evidence. A safe lifecycle defines an abstention or escalation path.

What to remember

• Design RAG as an ingestion flow joined to a serving flow by indexed chunks.
• Preserve source, freshness, and authorization metadata with every chunk.
• Retrieval finds candidate evidence; generation turns supplied evidence into an answer; grounding connects the answer’s claims to that evidence.
• RAG supplies knowledge at request time and does not replace model tuning.
• Plan for refresh, attribution, validation, and no-evidence behavior from the start.

Knowledge check

References

• Google Cloud, RAG Engine overview
• Google Cloud, Vertex AI APIs for building search and RAG experiences
• Google Cloud, Generative AI glossary: Retrieval-augmented generation
• Google Cloud, Develop a generative AI application
• Google Cloud, Check grounding with RAG
• Google Cloud, Set up data source access control
• Google Cloud, Fine-tune RAG transformations