Retrieval & RAG

A model only knows what it was trained on and whatever you put directly in the prompt. Retrieval-augmented generation - RAG - is how you bridge that gap: instead of retraining anything, you search your own documents for the relevant bits and hand those to the model alongside the question. LangChain's retrieval pieces exist specifically to make that pipeline manageable.

The five stages, in order

Every RAG setup, however fancy it eventually gets, is built from the same five stages:

Load - a document loader turns your source files (PDFs, web pages, database rows, whatever) into LangChain Document objects. Split - a text splitter cuts each document into smaller chunks, because embedding an entire 40-page PDF as one unit loses too much precision for retrieval to work well. Embed - an embedding model converts each chunk into a vector: a list of numbers positioned so that chunks with similar meaning end up close together in that space. Store - a vector store keeps those vectors somewhere you can search efficiently by similarity. Retrieve - given a new question, a retriever embeds the question the same way and returns whichever stored chunks are closest to it.

Building the pipeline

The first four stages run once, when you index your documents:

from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

# Document loaders now live in their own dedicated packages too - check
# the integrations page for whichever file type or source you're loading
# (PDF, CSV, a specific SaaS tool, etc.) for the current, maintained import
# rather than assuming it's still bundled in one place.
docs = load_documents("handbook.pdf")
chunks = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100).split_documents(docs)

vectorstore = Chroma.from_documents(chunks, embedding=OpenAIEmbeddings())
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

Then, at query time, the retriever plugs straight into an LCEL chain alongside your prompt and model:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableParallel, RunnablePassthrough

prompt = ChatPromptTemplate.from_template(
    "Answer using only this context:\n{context}\n\nQuestion: {question}"
)

rag_chain = (
    RunnableParallel({"context": retriever, "question": RunnablePassthrough()})
    | prompt
    | model
    | StrOutputParser()
)

answer = rag_chain.invoke("What's our policy on remote work?")

RunnableParallel runs the retriever and passes the original question through side by side, so the prompt template gets both the retrieved context and the question it needs to answer, in one step.

A current, practical note on which packages to use

Document loaders, embeddings, and vector store integrations used to mostly live in one large langchain-community package. That package was archived in mid-2026 and is no longer maintained - integrations have moved to dedicated packages like langchain-openai, langchain-chroma, or langchain-pinecone, each maintained separately. If a tutorial you're following imports a loader or vector store from langchain_community, it's worth checking that provider's current integrations page for its actual maintained package before you build on it.