Every guide before this one has described RAG as a fixed sequence: retrieve once, then generate. That works well for straightforward questions, but it breaks down on anything that genuinely needs more than one search - a question that has to be split into parts, a first retrieval that comes back thin, or a query that's phrased nothing like the document that answers it. Agentic RAG is what you get when you let the model decide, in the moment, whether one retrieval pass was actually enough.
What "agentic" adds to the pipeline
In standard RAG, retrieval is a step the pipeline always runs once, in a fixed order, regardless of the question. In agentic RAG, retrieval becomes a tool the model can call - it can decide to search, look at what came back, decide that's insufficient, reformulate the query, and search again, all before producing a final answer. This is the same tool-calling loop covered in the LangGraph and LangChain guides on this site, applied specifically to the retrieval step.
The model can loop back through retrieval with a reformulated query instead of committing to a single search attempt.
Query rewriting
The words a user types are often a poor match for the words in the document that actually answers them - "why did my payment fail" versus a document titled "Common Decline Codes and Their Meanings." Query rewriting has the model reformulate the user's question into one or more search-friendly queries before retrieval runs, closing that gap without requiring the user to phrase things better themselves.
Multi-hop retrieval
Some questions can't be answered by any single chunk, because the answer requires combining facts from different places - "which of our vendors located in the EU also appear in last quarter's compliance flags" needs one retrieval for EU vendors and another for compliance flags, then a step that reasons over both results together. Multi-hop retrieval is the pattern of running several dependent retrievals in sequence, where each search is informed by what the previous one returned, rather than trying to answer everything from one query.
Self-correction
The most distinctive agentic capability is deciding a retrieval attempt failed and trying again differently - noticing the returned chunks don't actually address the question, then either broadening the query, searching a different source, or, if nothing relevant turns up after a reasonable number of attempts, saying so explicitly instead of generating a confident-sounding answer from irrelevant context.