One wrong answer, two possible culprits
A RAG pipeline fails in exactly the way that makes it hard to debug: from the outside, a bad answer just looks like a bad answer. Read the response, see that it's wrong, and you still don't know why — whether the retriever handed the model the wrong documents, or handed it the right documents and the model ignored them anyway. Those are two completely different bugs, living in two different parts of your system, and a plain "was this a good answer" judgment can't tell them apart.
That ambiguity is the entire reason RAG evaluation exists as its own discipline rather than a special case of general LLM evaluation. A standard input/output judge treats the system under test as a black box — prompt goes in, response comes out, score the response. That's a fine model when the only thing between the prompt and the answer is the language model's own reasoning. RAG breaks the assumption by inserting a retrieval step in the middle, and that step fails in ways that have nothing to do with how well the model reasons.
Ragas exists to pull those two failure surfaces apart. Before you can appreciate why it defines four separate metrics instead of one overall score, you need to see exactly how the same wrong answer can come from opposite ends of the pipeline — and why that distinction is the one thing a single score will never give you.
Did it hallucinate, or did it never get the facts?
Picture two runs of the same RAG system, both producing the same wrong answer to the same question. In the first, the retriever does its job — the right documents are sitting right there in the context window — but the model writes something the documents don't actually support. In the second, the retriever comes up empty or pulls the wrong documents entirely, and the model does its best with what it was given, which was never going to be enough. Read only the final answer, and these look identical. Read the pipeline, and they're opposite failures with opposite fixes.
The original Ragas paper (Es et al., 2023) frames this precisely: a RAG system has at least two independently-failing components, the retriever and the generator, and traditional end-to-end evaluation collapses both into a single judgment that structurally cannot distinguish which one broke. That's not a minor omission — it's the reason a team can stare at a low quality score for weeks without knowing whether to touch their embedding model or their prompt.
The clean way to hold this in your head: did the model hallucinate, or did it never get the right information to begin with? Every metric this course covers exists to answer one half of that question, and reading them together is what finally lets you answer it with confidence instead of a guess.
The catch: it's tempting to assume a wrong answer is usually a generation problem, because the model is the visible part of the system doing the "thinking." In practice a huge share of RAG failures are retrieval failures wearing a generation costume — the model answered reasonably given what it had, and what it had was wrong.
Why a judge model alone doesn't reach this problem
If you've already built out a general LLM evaluation practice — LLM-as-judge scoring, rubrics, regression suites in CI — it's fair to ask why any of that isn't already sufficient. The answer is that a general judge model, however well-calibrated, is still scoring the same black box: prompt in, response out. It has no visibility into what got retrieved, whether it was ranked well, or whether it actually contained what the question needed. It can tell you the final answer felt wrong. It can't tell you where in the pipeline that wrongness originated.
This matters because retrieval has its own failure modes that have nothing to do with the language model's reasoning quality at all. A query that doesn't phrase things the way your corpus was chunked. An embedding model that places semantically related but factually different passages right next to each other in vector space, so the "closest" match is confidently wrong. An index that returns the five most similar chunks when the actual answer lives in the sixth, just outside the cutoff. None of that shows up if the only thing your evaluation ever looks at is the finished response.
So a general eval practice isn't wrong, exactly — it's incomplete for this specific architecture. It's built to judge reasoning over what a model already has. RAG adds a whole upstream stage that decides what the model has in the first place, and that stage needs its own instrumentation, not a more careful reading of the final output.
The catch: stacking a stronger judge model on top of the same black-box view doesn't fix this — a better judge is still only looking at the final answer. What's missing isn't judgment quality, it's visibility into a pipeline stage the judge was never shown in the first place.
Four numbers instead of one
Ragas' answer is to refuse to collapse the pipeline into a single score at all. It defines two metrics that grade what the retriever handed back, and two that grade what the generator did with it — a direct, structural response to the two-culprit problem from a moment ago. Get four numbers instead of one, and two of them answer "did the retriever do its job," while the other two answer "did the generator do its job given what it actually received."
That split does something a blended score never could: it makes disagreement between the numbers informative rather than confusing. A low score on the generation side paired with strong retrieval scores is close to a direct pointer at your prompt or your model. A low score on the retrieval side, no matter how the generation metrics look, tells you the problem lives upstream, in your index or your chunking or your embedding choice, before the model ever gets a chance to reason about anything.
That's the shape of the entire toolkit this course builds toward — not one verdict, but four independent readings that, taken together, triangulate which half of the system actually needs your attention. The next lesson gets specific about what each of those four numbers actually measures and how each one is computed.
The instrument you need has to see both stages
The core idea to carry forward is simple even though its consequences aren't: a RAG pipeline has two places to fail, retrieval and generation, and they produce the same symptom — a wrong answer — through completely different mechanisms. Any evaluation approach that only looks at the final response is structurally blind to which one actually happened, no matter how sophisticated the judge scoring it the final text.
Ragas' whole design follows from taking that seriously. Rather than one holistic quality score, it splits the pipeline in half and grades each side on its own terms — a direct, deliberate answer to a problem that generic LLM evaluation was never built to solve, because generic evaluation doesn't know a retrieval step exists.
In the next lesson, we'll get concrete about what that split actually looks like: the four metrics themselves — faithfulness and answer relevancy on the generation side, context precision and context recall on the retrieval side — and exactly how each one is computed, so a low score stops being a mystery and starts being a lead.