Every RAG system, regardless of which tool or framework builds it, breaks down into the same six stages. Two happen once, ahead of time, to prepare your documents. Two happen every single time someone asks a question. Knowing which is which changes how you think about performance and cost.
Stage 1-3 happen once: indexing
Load pulls your source documents in from wherever they live - PDFs, web pages, a database, a wiki export - and turns them into plain text your pipeline can work with. Split (also called chunking) breaks that text into smaller pieces, because embedding and retrieving a 40-page PDF as one unit doesn't work - you need pieces small enough to be individually relevant. Embed runs each chunk through an embedding model, which converts it into a vector: a list of numbers that captures its meaning, positioned so that semantically similar chunks end up near each other in that numerical space.
# Rough shape of the indexing stage (framework-agnostic pseudocode)
documents = load("policies/") # Stage 1: Load
chunks = split(documents, chunk_size=1000) # Stage 2: Split
vectors = [embed(chunk) for chunk in chunks] # Stage 3: Embed
vector_store.upsert(chunks, vectors) # Stage 4: Store
Store saves those chunks and their vectors into a vector database, indexed for fast similarity search. This whole load-split-embed-store sequence is called indexing, and the defining thing about it is that it runs ahead of time, in a batch job, whenever your source documents change - not on every user request. Get this part wrong or skip it and nothing downstream can save you, which is why the next two guides in this series go deep on splitting and storage specifically.
Stage 5-6 happen every request: querying
Retrieve is what runs when a real question comes in: embed the question using the same embedding model, search the vector store for the chunks whose vectors are closest to it, and pull back the top handful (often called "top-k," where k is usually somewhere between 3 and 10). Generate hands those retrieved chunks to the LLM along with the original question, typically wrapped in a prompt template along the lines of "using only the following context, answer this question," and the model produces the final response.
# Rough shape of the query stage - runs on every request
question_vector = embed(user_question) # Stage 5a
top_chunks = vector_store.search(question_vector, k=5) # Stage 5b: Retrieve
answer = llm.generate(prompt(question, top_chunks)) # Stage 6: Generate
Where the real engineering effort goes
| Stage | When it runs | What usually goes wrong |
|---|---|---|
| Load | Once per document change | Messy extraction - tables and formatting turning into garbled text |
| Split | Once per document change | Chunks too big (noisy) or too small (context-less) - covered next guide |
| Embed | Once per document change | Rarely the problem - mostly a model choice, not a design decision |
| Store | Once per document change | Picking a vector database that doesn't fit your scale or metadata needs |
| Retrieve | Every request | The wrong chunks get pulled back - top-k too small, or a bad similarity match |
| Generate | Every request | The model ignores the retrieved context, or the prompt template buries it |
In practice, the two stages that consume the most iteration time are Split and Retrieve - Split because chunk boundaries directly determine what's even possible to retrieve later, and Retrieve because "did we actually pull back the right chunks" is the single most common failure mode in a RAG system that otherwise looks correct end to end.