The discipline that separates demos from products
A demo needs to work once, in front of you, on the examples you happened to try. A product needs to keep working across millions of requests you'll never personally see, on inputs you didn't anticipate, after the underlying model provider silently changes something upstream. The gap between those two is entirely made of evaluation and observability — the practices that tell you whether the system is actually working, not whether it felt like it was working the last time you glanced at it.
This is a harder problem than testing conventional software. A unit test compares output to one correct answer. An LLM rarely has one correct answer — "explain this concept" has thousands of acceptable phrasings and a much smaller but nonzero number of subtly wrong ones. You can't write a simple equality check and call it done. You need a different kind of instrument.
Traditional software fails loudly: a null pointer exception crashes, a 500 status code shows up in a dashboard. LLM systems fail quietly — the server returns 200, the response is well-formed, coherent, grammatically perfect, and wrong. Building that instrument starts with the cheapest, most reliable layer of measurement, and works up from there.
Comparing against an answer key
The oldest instinct in evaluation is to compare generated text against a reference answer — the way you'd grade a translation or a summary against a known-good version. Metrics like BLEU and ROUGE score n-gram overlap between the output and the reference; exact match just checks for strict string equality. It's the same intuition as a spelling test: here's the correct answer, tell me how close you got.
That intuition breaks down fast once you're evaluating open-ended generation. Two semantically identical answers with different phrasing will score low on n-gram overlap even though they're both correct, while a subtly wrong answer that happens to share vocabulary with the reference can score deceptively high. The metric is measuring word overlap, not correctness — and for most LLM tasks those two things come apart quickly.
So reference-based metrics aren't obsolete, they're just narrow. They're genuinely useful for structured extraction, classification, and translation — anywhere a reference answer is truly singular, where there's essentially one right way to phrase the correct output. Chat, reasoning, and open-ended generation are exactly where they stop telling you anything reliable, which is why most eval pipelines lean on other layers for that territory.
The catch: A subtly wrong answer that happens to share vocabulary with the reference scores high on BLEU or ROUGE — the metric can't tell fluent-and-wrong from correct. Reserve these metrics for tasks with a genuinely canonical output, and skip them for anything open-ended.
Catch what doesn't need judgment
Before you spend a single token asking a model to judge another model's output, ask a simpler question: does this output even have the right shape? Does the JSON parse? Does it match the expected schema? Does a required field actually contain an enum value from the allowed set? None of that requires an opinion — it requires a parser.
This is why deterministic checks come first in the layered discipline of evaluation. Schema validation, regex and format checks, unit-testable assertions on structured fields — these are cheap to run, completely free of judge bias, and they catch an entire class of failure that has nothing to do with quality in the fuzzy sense: malformed output, missing required fields, a wrong enum value, a date in the wrong format. A model that writes beautifully but returns invalid JSON has still failed, and you don't need an LLM to tell you that.
Running these checks first isn't just tidiness, it's economics. Every output that fails a schema check is an output you don't need to pay judge-model tokens to evaluate — the deterministic layer acts as a free filter that narrows what actually needs the more expensive, more subjective layer of scoring.
The catch: Deterministic checks only catch what can be checked deterministically — they'll wave through a response that's perfectly valid JSON and completely wrong in substance. They're a floor, not a ceiling, on quality.
Scoring one answer versus choosing between two
Once you're past what deterministic checks can catch, you're left with the hard part: judging quality that can't be reduced to a boolean. The dominant approach is LLM-as-judge — a second model reads the output and produces a score — but "produces a score" hides an important design decision about how that judge is asked to work.
Pointwise scoring shows the judge a single output and asks it to rate that output against a rubric: a 1-to-5 scale, a pass/fail call, or a labeled dimension like faithfulness. Pairwise scoring shows the judge two outputs side by side — a new prompt version against a baseline, say — and asks it to pick the better one, or declare a tie. Same underlying instrument, two very different questions being asked of it.
The tradeoff comes down to what you're actually trying to learn. Pointwise scores are absolute and comparable across time, which is exactly what regression testing and dashboards need — you want a number today that means the same thing as the number from three months ago. Pairwise comparisons are more reliable per-judgment, because judges are demonstrably better at relative calls than absolute ones, but a pairwise result doesn't give you a stable trend line unless you're comparing against the same fixed baseline every single time.
In practice, you reach for pointwise when you're building a CI gate or a dashboard that needs to track drift over months, and pairwise when you're actively tuning — is this new prompt variant better than the one it's replacing, right now, today. Both approaches share the same weak point, though: they depend entirely on the rubric the judge is given, and a vague rubric produces a vague, unreliable score no matter which mode you use.
The tradeoff: Pairwise judgments are more reliable per comparison, but they don't give you a stable number to track over months without a fixed baseline to compare against every time. Pointwise gives you that trend line, at the cost of the judge being worse at absolute calls than relative ones.
A single score hides the failure
Say a judge hands back a 3 out of 5. What actually went wrong? Was the answer factually incorrect, incomplete, badly formatted, or just off in tone? A single holistic quality score can't tell you, because it's already collapsed several independent failure modes into one number before you ever see it.
That's the case for rubric design: instead of asking a judge for one vague "quality" score, you decompose "good answer" into named, independently scorable criteria — correctness, completeness, tone, format adherence, whatever dimensions actually matter for the task. Each dimension gets scored on its own, so a regression in one doesn't get averaged away by strength in another.
The payoff shows up the moment something breaks. A decomposed rubric tells you which dimension regressed — did correctness drop, or did completeness drop while correctness held steady — and that's the information that actually points you toward a fix. A single 3/5 tells you something is wrong; a rubric tells you what.
Good rubric criteria are specific enough that two different judge runs would produce the same score for the same output, which is the same standard you'd hold a human rubric to. Vague criteria like "is this a good response" invite inconsistency regardless of whether a human or a model is applying them — the discipline of writing a rubric is really the discipline of deciding, in advance, exactly what you're going to hold the system accountable for.
The catch: A single holistic score conflates failure modes — you can't tell from a 3/5 whether the answer was wrong, incomplete, or just badly formatted. Named, independently-scorable criteria are what actually drive a fix when something regresses.
A layered discipline, not a single tool
What ties these four pieces together is that no single layer is sufficient on its own. Reference-based metrics are precise but narrow. Deterministic checks are cheap and reliable but only catch shape, not substance. Judge scoring — pointwise or pairwise — catches quality that can't be reduced to a boolean, but only as well as the rubric behind it is written. Run them in that order: deterministic first, because it's free; reference-based where a genuine answer key exists; judge scoring, guided by a decomposed rubric, for everything else.
None of this is optional overhead bolted onto a working system — it's the actual mechanism by which you find out whether the system is working at all, given that failures in this domain are quiet by default and won't announce themselves.
In the next lesson, we'll turn the lens on the judge itself. LLM-as-judge is only as trustworthy as the model doing the judging, and that model has its own failure modes — position bias, verbosity bias, and self-preference bias — that can quietly distort every score it produces.