Retrieval doesn't have to happen once
Every retrieval pattern we've covered so far shares a hidden assumption: you retrieve once, you get some chunks, and you hand them to the language model. That assumption is baked into most RAG pipelines, and for simple questions it works fine. But complex questions — the kind that require synthesizing multiple sources, following a chain of reasoning, or verifying a claim against evidence — tend to fall apart under that single-pass model.
The patterns in this lesson break that assumption entirely. Instead of treating retrieval as a one-time lookup, they treat it as a process that can loop, reflect, and correct itself. The model doesn't just consume retrieved context — it decides when to retrieve, what to retrieve next, and whether what it found is actually useful.
There are five patterns here, and they exist on a spectrum from research curiosity to the thing most production agentic systems actually run. Understanding all five will help you know which one to reach for — and which ones to avoid in production.
Teaching a model to ask before it retrieves
Self-RAG is the most architecturally ambitious of these patterns. The core idea is to fine-tune the language model itself to generate special reflection tokens inline with its output — tokens that express meta-level decisions about retrieval and grounding.
A Self-RAG model produces four kinds of reflection tokens as it generates. A [Retrieve] token signals that the model has decided it needs external information before continuing. [Relevant] and [Irrelevant] tokens let the model assess each retrieved chunk — essentially annotating whether what came back is actually useful for the current generation. And [Supported] and [Unsupported] tokens let the model self-critique whether the claim it just made is grounded in the retrieved evidence.
What makes this powerful is the dynamism. A traditional RAG pipeline retrieves on every query, whether it needs to or not. Self-RAG retrieves only when the model explicitly decides it's necessary — and then evaluates the quality of what it got. The self-critique loop is what reduces hallucination: the model isn't just generating, it's checking its own work.
The practical reality: Self-RAG requires fine-tuning. You can't take a closed model like GPT-4 or Claude and add reflection tokens — the behavior has to be trained in. That constraint alone keeps Self-RAG largely in research settings rather than production stacks. The concept of LLM-controlled retrieval is highly influential, but the implementation requires infrastructure most teams don't have. ReAct, which we'll cover shortly, gives you most of the same benefits without the fine-tuning requirement.
Pause when you're uncertain, look it up, continue
FLARE — Forward-Looking Active Retrieval — takes a more mechanical approach to the same problem Self-RAG solves with fine-tuning. Rather than training the model to know when it needs help, FLARE watches the model's token probabilities and intervenes when confidence drops below a threshold.
The flow is this: the model starts generating text normally. At each token, FLARE checks the predicted probability. When it detects a low-confidence token — a token the model is uncertain about — it stops generation, forms a retrieval query from the text generated so far, retrieves relevant documents, and then restarts generation from that point with the new context in hand.
Think of it like writing a research essay when you hit a gap in your knowledge. You don't write through the uncertainty hoping for the best — you pause, look it up, and then pick up where you left off. FLARE is that pause mechanism, automated.
The catch: Confidence calibration is harder than it sounds. Language models aren't reliably calibrated — a model can be confidently wrong, producing high-probability tokens that are factually incorrect, and FLARE won't trigger. It can also be uncertain about stylistic choices that have nothing to do with factual accuracy. The iterative stop-retrieve-continue cycle also adds significant latency: on a long response, you might halt and retrieve a dozen times. In practice, FLARE tends to underperform ReAct for complex tasks, and its latency profile makes it difficult to deploy at scale.
The loop that actually ships to production
ReAct is the pattern you'll encounter in almost every serious production agentic RAG system. It doesn't require fine-tuning. It works with any capable language model. And it's expressive enough to handle multi-step reasoning problems that completely defeat single-pass retrieval.
The structure is a loop with three phases: Thought, Action, and Observation. In the Thought phase, the model reasons about what it knows so far and what it needs to find out. In the Action phase, the model selects a tool and issues a query — and retrieval is just one tool among many alongside web search, SQL execution, or code interpretation. In the Observation phase, the model reads the result and updates its understanding. That loop repeats until the model decides it has enough information to produce a final answer.
What makes ReAct powerful in practice is how naturally it handles questions that require sequential reasoning. Consider a question like "what's the current price of the stock for the company that makes the most-used database in the world?" A single embedding search can't answer that — it requires knowing what the database is, identifying the company behind it, and then fetching current financial data. ReAct handles each step as a distinct action, each building on the last.
LangGraph, LlamaIndex Workflows, and similar frameworks are essentially scaffolding for ReAct loops. When you build an agent that can call tools and reason across multiple steps, you're building ReAct — whether or not you use that name for it.
The tradeoff: Loops introduce latency and cost. Each Thought-Action-Observation cycle is an LLM call, and complex queries can require four, five, or more iterations. You also need to budget for the case where the agent loops unnecessarily or gets stuck in an unproductive reasoning path — so most production systems implement a maximum iteration count and a fallback.
Generate, find the gaps, retrieve again
Iterative retrieval is a more structured, less free-form version of what ReAct does. Where ReAct lets the model reason flexibly and decide what action to take next, iterative retrieval follows a fixed pattern: generate a partial answer, explicitly identify what's missing, form queries targeting those gaps, retrieve, and regenerate with the fuller context.
The critical difference from ReAct is the gap analysis step. The model isn't just reasoning about what to retrieve next — it's systematically cataloguing what the current partial answer doesn't yet address. That explicit accounting makes iterative retrieval more predictable and easier to debug than ReAct. You can inspect the gap list at each iteration and understand exactly what the system thought it was missing.
This pattern earns its place in research agents and complex report generation pipelines, where the goal is comprehensive coverage of a topic rather than a quick point answer. If you're building a system that generates technical summaries, due diligence reports, or literature reviews, iterative retrieval's methodical approach is often a better fit than the more fluid ReAct loop.
When the answer to one question is the query for the next
Multi-hop retrieval handles a specific class of problem where the answer can't be found in a single search — where you need to chain retrievals, each one building on what the previous one returned. The result of hop N becomes the query for hop N+1.
The canonical example makes the structure concrete: "What is the net worth of the CEO of the company behind the most popular JavaScript framework?" No single document answers that. You need hop one to establish that the most popular JavaScript framework is React, hop two to identify that React is maintained by Meta and that its CEO is Mark Zuckerberg, and hop three to retrieve his current net worth. Each hop is a retrieval call, and each call is only possible because of what the previous one returned.
IRCoT — Interleaved Retrieval with Chain-of-Thought — is the cleanest published implementation of this pattern. It interleaves the chain-of-thought reasoning steps with retrieval calls, so the model's reasoning and its evidence-gathering are synchronized at each step rather than separated into a plan-then-retrieve structure.
The compounding problem: Each hop multiplies latency — three hops means three sequential retrieval calls, and you can't parallelize them because each depends on the last. More critically, errors compound: if hop one returns the wrong answer, every subsequent hop is building on that mistake, and the final answer can be confidently wrong in a way that's hard to detect. Multi-hop systems need robust answer validation at each step and clear fallback behavior when a hop returns ambiguous or conflicting results.
Let the question determine how many times you retrieve
The through-line across all five patterns is a shift in how you think about retrieval's role. It isn't a lookup — it's a capability the model can invoke, repeat, and reflect on. Self-RAG makes that explicit through training. FLARE makes it automatic through confidence monitoring. ReAct makes it composable through tool use. Iterative retrieval makes it systematic through gap analysis. Multi-hop makes it chainable through sequential dependency.
For most teams building production systems today, ReAct is where to start. It requires no fine-tuning, works with every capable model, and is well-supported by existing agent frameworks. Add multi-hop reasoning when your questions genuinely require it — and build in iteration limits and error handling before you ship.
In the next lesson, we'll move beyond text entirely and look at multimodal retrieval — how to build systems that can retrieve across images, documents, and other non-text content using models like CLIP and ColPali.