LLM-as-Judge

A platform team wires an LLM judge into its pull-request pipeline: every diff gets a rubric score before a human reviewer opens it, on the theory that the model can triage load off senior engineers before it ever reaches them. The first version of the rubric asks the judge to rate whether the change is “well-implemented and appropriate for the codebase.” Every diff scores between seven and nine out of ten. The judge is not malfunctioning — it has been asked to grade against a criterion no procedure could fail. An adjective is not an instruction, and a model asked to be honest about a vague standard defaults to being generous.

The team tightens the wording twice more, and the scores stay clustered near the top until someone notices what actually moved the needle: diffs that included a test file scored measurably higher than diffs that didn’t, regardless of whether the new test covered the change. The rubric had quietly learned to reward the presence of a pattern rather than the correctness the team meant to grade. This is the ordinary condition of standing up an LLM judge, not an edge case. Chapter 9 of Volume VI covered this technique at fleet scale — using a judge to pick a winner among several agents’ attempts at the same ticket — and reported the finding, from Anthropic’s own production system, that a single well-specified rubric call can outperform an elaborate multi-judge setup. That finding assumes the rubric is well-specified. Getting there is a craft with its own failure modes, three of which this chapter is about: rubrics vague enough to reward the wrong thing, judges that quietly favor output shaped like their own, and judges whose calibration moves out from under a team that never checked.

What makes a rubric gradeable

Anthropic’s own cookbook for building evaluations lays out three ways to grade a model’s output: code-based grading, which it describes as fast and highly reliable wherever a pattern can be matched; human grading, the most capable and also the slowest and most expensive; and model-based grading, positioned as the practical middle ground for judgments a human would make but a team cannot afford to make at volume. The catch sits inside the third option. A model-based grader is only as good as the procedure it has been handed, and a procedure written as an adjective — helpful, well-implemented, appropriate — is not a procedure. It is a description of what a good outcome feels like, left for the judge to operationalize on its own, inconsistently, one call at a time.

The fix is mechanical rather than clever: replace every adjective in the rubric with the steps a careful human reviewer would actually perform to check it. “Well-implemented” becomes a checklist — does the diff touch only the files the ticket named, does every new code path have a corresponding test, does the change pass the existing suite. “Appropriate for the codebase” becomes a specific comparison — does the naming convention match a handful of named examples from the same directory. Neither question asks the judge to have taste. Each has a checkable answer, which is the property that turns a rubric from a mood into a rubric.

A framework for rubric specificity

The biases baked into the grade

Even a well-operationalized rubric inherits the biases of the model executing it, and the foundational LLM-as-judge study named three specifically. Zheng and colleagues, evaluating GPT-4 as a judge against human preference data in 2023, documented position bias — a judge’s preference shifting measurably depending on which candidate is presented first — and verbosity bias, the tendency to score a longer answer higher independent of whether the extra length adds anything. Both are addressable with mechanical fixes a team can build once and stop thinking about: randomize or swap presentation order and average the result, and instruct the rubric explicitly that concise and verbose answers of equal correctness score the same, since a judge never told that length is neutral will default to reading it as evidence of effort.

The third bias Zheng’s study named is harder to fix with a formatting rule, because it is not about how the candidates are presented — it is about who wrote them. Self-enhancement bias, more often called self-preference bias in the research that followed, is a judge’s tendency to rate an output more highly because it resembles the judge’s own writing, independent of whether a differently styled answer would have been equally or more correct.

When the judge prefers itself

Panickssery, Bowman, and Feng gave the phenomenon its clearest demonstration in 2024: across several model families, an LLM asked to compare its own generations against a competitor’s rated its own work higher than human annotators — who judged the two outputs roughly equivalent — did. More strikingly, the size of that self-preference gap correlated linearly with how well the model could recognize its own writing in the first place; fine-tuning a model to get better at telling its own outputs apart from everyone else’s made it a more self-preferential judge, not a more honest one. Self-recognition and self-preference move together, which suggests the bias is not a quirk to patch but a structural consequence of the same capability that makes a model useful as a judge at all.

Fine-tuning a model to better recognize its own writing made it a more partial judge, not a more honest one.

On Panickssery, Bowman & Feng, 2024

A follow-up from Wataoka, Takahashi, and Ri the same year reframes what is actually being measured. Testing GPT-4 as a judge, they found the bias tracks perplexity: the judge consistently rates text it finds more predictable — more familiar in its own statistical sense — higher than text it finds surprising, whether or not that text was self-generated. Self-preference, on this account, is really familiarity-preference; a model happens to find its own phrasing the most familiar thing it reads, so “prefers itself” is the visible symptom of a judge quietly scoring how expected the prose sounds rather than how correct the content is. The distinction matters operationally: a judge run from the same model family as one of the candidates it is grading carries this bias structurally, before a single word of the rubric is written, and no amount of rubric craft removes it. The available mitigation is architectural, not linguistic — run the judge on a different model, ideally a different provider, from anything it is asked to grade. Operon’s own Arena judge, covered in Volume VI, sidesteps part of the problem by scoring structured metrics rather than raw diffs — but a structured input format doesn’t remove self-preference if the judge model and one of the graded agents still share a lineage; the fix there is still an independent judge, not a different shape of input.

A judge that quietly changes its mind

There is a fourth failure this chapter has not named yet, because the research base under it is thinner than the other three: a rubric and a judge that scored consistently last quarter producing a visibly different distribution of scores today, with no change to the rubric, the candidates, or the team’s process. Practitioners who run LLM judges in production describe this often enough that it has an informal name — judge drift — but it does not yet have the kind of dedicated, replicated research literature that backs position bias or self-preference. What exists is adjacent and corroborating rather than definitive, and this chapter treats it that way rather than dressing it up as more settled than it is.

A 2026 large-scale study out of UC Berkeley, evaluating LLM judges for agreement, consistency, and bias across twenty-one models, found that high test-retest reliability — the same judge scoring the same input the same way twice — routinely coexisted with severe, undetected bias, and flagged in its own limitations that hosted judge models “are known to drift across provider-side updates, sometimes silently.” That is a narrower and more honest claim than “judges drift” stated as a general law: it says a judge can be internally consistent within a measurement window and still be standing on a foundation — a hosted model version — the provider is free to change without telling anyone. A separate 2026 study of LLM-designed rubrics found something structurally similar from another angle: a model applies its own rubric consistently to itself, but that consistency degrades in factual, knowledge-intensive grading and doesn’t transfer well across model families. Consistency, in both studies, turns out to be a weaker guarantee than it sounds — a judge can be reliable and wrong at the same time, and reliability alone will not tell a team which.

Treat judge drift, then, as an observed operational risk worth guarding against, not as an established, quantified phenomenon with a citation for how much or how often it happens. The honest answer, at time of writing, is that nobody has published one.

Building a judge worth trusting

1. Operationalize every adjective in the rubric before it ships. If a human reviewer couldn’t explain the check in one sentence without using a quality word, the judge can’t execute it consistently either.
2. Neutralize position and length explicitly — randomize candidate order and state directly that concise and verbose answers of equal correctness score the same.
3. Never let a judge grade a candidate from its own model family without a second, differently sourced check. Self-preference bias is structural, not a prompting mistake, and no rubric wording fixes it.
4. Calibrate against a small human-graded set before trusting the judge at any volume, and keep that set on hand — it is the only way to notice, later, that the judge’s agreement with human judgment has moved.
5. Re-run the calibration set on a schedule, not just once at launch. The rubric didn’t change; the model behind the judge might have, and a team usually finds out from a bad decision before it finds out from a dashboard.

None of this replaces the work Chapter 2 described — a judge is still one piece of grading logic inside an eval, and grading logic is still software with its own bugs, worth reading behind rather than trusting on sight. What this chapter adds is upstream of that discipline: the judge has to be built well before its scores are worth reading transcripts to check. A vague rubric, a judge sharing its lineage with the thing it grades, and a calibration nobody has rechecked since launch are three different ways to end up trusting a number that was never earned.