An AI Agent node is only as good as what it knows. Out of the box, a model only knows what it learned in training - nothing about your product docs, your support history, or your internal wiki. RAG (retrieval-augmented generation) fixes that by giving the agent a way to look things up before it answers.
The two-phase pattern
RAG in n8n happens in two separate workflows. The indexing phase runs once (or periodically): load your documents, split them into chunks, convert each chunk into a numeric representation called an embedding, and store those embeddings in a vector store. The retrieval phase runs every time the agent needs an answer: take the user's question, embed it the same way, find the stored chunks whose embeddings are most similar, and hand those chunks to the model as context before it responds.
Building the indexing workflow
A typical setup: a trigger (manual, or scheduled for periodic re-indexing) feeds documents into a Document Loader node, then a Text Splitter node breaks long documents into smaller chunks (a few hundred words each, with some overlap between chunks so context isn't lost at the boundaries), then an Embeddings node (commonly OpenAI's embedding models, though other providers work too) converts each chunk to a vector, and finally a Vector Store node writes everything to storage.
Picking a vector store
n8n supports several vector store nodes, including Pinecone, Supabase (pgvector), Qdrant, and an in-memory option for quick testing. In-memory is fine for prototyping but resets every time the workflow restarts - for anything real, use a persistent store. Pinecone and Qdrant are common managed choices; Supabase's pgvector is a good fit if you're already running Postgres and want to avoid adding another service.
Wiring retrieval into the AI Agent
Add a Vector Store node configured as a "retriever" and connect it as a tool input on the AI Agent node from the earlier guide. The agent then decides, based on the conversation, whether it needs to search your documents - it's just another tool in its toolbox, alongside anything else you've given it access to.
Chunk size and retrieval quality
Chunks that are too large dilute relevance (the model gets a lot of irrelevant surrounding text); chunks too small lose context (a fact gets separated from the sentence that explains it). A few hundred tokens per chunk with roughly 10-20% overlap between consecutive chunks is a reasonable starting point - then adjust based on whether the agent's answers are missing context or getting distracted by noise.