A trustworthy judge still needs a test to run

You spent the last lesson getting your LLM judge calibrated against human raters — checking it agrees with people often enough to trust its scores, tightening the rubric where it didn't. That was the hard part, or so it feels. But a calibrated judge that only ever gets pointed at whatever example you happen to be looking at that afternoon isn't actually evaluating your system. It's evaluating your mood.

The gap between "I tried a few examples and it seemed fine" and "I know this works" isn't better prompting or a smarter judge. It's repeatability. You need the same fixed set of inputs run through the same scoring process every time, so that a score today means the same thing as a score next week — and so a change to your prompt shows up as a number moving, not a vague feeling that something's different.

That means two things you don't have yet: a dataset worth trusting, and a tool to actually run it with. We'll cover the landscape of frameworks teams reach for first, then build the golden dataset that gives those frameworks something real to chew on, and finally turn that combination into a regression suite you re-run every time something changes.

Nobody hand-rolls an eval harness twice

The first time you need to run a hundred test cases through a judge and get scores back, it's tempting to just write a for-loop and call it a day. The second time — when you need CI integration, dataset versioning, and a way to show a stakeholder a regression trend — you'll wish you'd picked a framework instead.

The landscape splits along a fault line: development-time libraries versus production-time platforms, and no single tool does both well. RAGAS is the one to reach for if you're evaluating a RAG system specifically — it ships published, RAG-aware metrics like context precision, context recall, and faithfulness, backed by an academic methodology rather than a vibe. DeepEval is the broadest general-purpose option, and its killer feature is that it integrates as pytest-style assertions, so an eval failure looks exactly like a test failure to your CI system. promptfoo takes a different angle entirely — it's CLI-driven and built for testing a matrix of prompts against a matrix of models, and it doubles as a red-teaming tool for security test cases.

Then there's the hosted side. Braintrust gives you regression tracking, human annotation queues, and dashboards a stakeholder can actually look at without touching a terminal. Confident AI is a hosted layer built on top of DeepEval for the same reason. TruLens leans instrumentation-first, coupling tightly to tracing, and MLflow's LLM evaluate module is the right call if your team already lives inside MLflow's experiment tracking for everything else.

The pattern that emerges once teams have run this for a while: they don't pick one tool, they pick two. A lightweight, CI-friendly library — DeepEval, RAGAS, or promptfoo — handles pre-merge gating, fast and cheap enough to run on every pull request. A platform built for annotation queues and regression history — Braintrust, LangSmith, Arize — handles the ongoing production side, where you need dashboards and trend lines, not just a pass or fail. Picking the right framework matters, but it's a hollow exercise without the dataset to point it at — which is where we're headed next.

Turning vibes into a suite

A golden dataset is what turns "I tried a few examples and it seemed fine" into an actual number: a curated, versioned, hand-checked collection of representative inputs, each with an expected output, a label, or a scoring rubric attached. The word "curated" is doing real work there — this isn't a random sample, it's a deliberately assembled set that someone has reviewed and vouched for.

Where do the examples come from? Ideally, real production traffic — sampled and reviewed, not synthetic guesses — because real usage is reliably a different distribution than what you imagined users would ask. An eval suite built entirely from inputs you dreamed up in a planning meeting tests what you assumed people would do. One built from actual logs tests what they actually do, and those two sets of questions rarely overlap as much as you'd hope. Before launch, when there's no production traffic yet, synthetic generation — an LLM producing plausible inputs, human-reviewed before they go in — fills the gap reasonably well as a starting point.

A useful starting size is smaller than most people expect: 50 to 100 examples for a single feature is enough to catch most regressions, and going bigger before you've validated the suite is doing its job is often wasted effort. Split that roughly 40 to 60 core, happy-path cases and 15 to 25 deliberately adversarial edge cases — the malformed inputs, the ambiguous phrasing, the cases you already know are hard. The edge cases are where regressions actually hide; the core cases mostly just confirm you haven't broken anything catastrophic.

Once you have this set, it becomes the fixed instrument you measure every future change against — which is exactly the point, and exactly what makes regression testing possible in the first place.

The suite that catches what you can't see

Here's the failure mode that regression testing exists to catch: you tweak a prompt to fix one specific case, it works, you ship it — and three unrelated cases silently get worse. Nothing crashes. Nothing throws an error. The system just quietly does a worse job on inputs you weren't looking at when you made the change. That's the quiet-failure problem, and it's the default behavior of LLM systems, not an edge case.

Regression testing means re-running that fixed golden-dataset suite every time a prompt changes, a model version changes, or a retrieval configuration changes, and comparing the new score against the last known-good baseline. The mechanism is simple — it's the discipline of actually doing it, every time, that's hard. A regression suite is the only thing standing between "seemed fine when I tried it" and actually knowing whether that's true.

The dataset itself isn't static either. It needs to be refreshed periodically from new production samples, or it calcifies into testing what users asked for six months ago instead of what they're asking now. But refreshing it introduces its own subtlety: once the dataset changes, last month's score and this month's score aren't directly comparable anymore unless you know which dataset version each one was run against. So the dataset needs versioning too — the same discipline you're applying to prompts and models, just one level up the stack.

Put those two things together — a fixed suite, run automatically, compared against a versioned baseline — and you have something that actually catches regressions instead of just hoping you'd notice.

A suite nobody's forced to run doesn't count

Everything in this lesson only works if someone actually runs it. A golden dataset that exists but that someone has to remember to evaluate against isn't really a suite — it's a manual step waiting to be skipped the first time a deadline gets tight. And deadlines are exactly when regressions are most likely to slip through, because that's when people are moving fastest and checking least.

The fix isn't a better reminder or a more diligent team. It's making the suite structurally impossible to skip — which is what turns eval discipline from an occasional good habit into an enforced one.

In the next lesson, we'll take the golden-dataset suite you just built and put it behind an automatic gate that blocks merges — CI quality gates and prompt versioning, so the suite runs whether anyone remembers to or not.