Three infrastructure bugs and an eval suite that did not see them

In a postmortem published on 17 September 2025, Anthropic wrote that its evaluations simply didn't capture the degradation users were reporting, and that it had relied too heavily on noisy evaluations.[Anthropic] The stated causes were three overlapping infrastructure bugs between 5 August and 18 September 2025: a context window routing error, output corruption, and a TPU miscompilation. None of them was a weight change.

That last sentence is the whole reason this incident is worth reading. It is the strongest published example of a team with a serious eval suite, looking directly at the system, not seeing something real.

What the operator actually said

This one is an operator account, which makes it different from most items in the failures category. Anthropic published the causes itself, named three separate faults, and described its own evaluation practice as part of why detection was slow.

Two phrases from it do the work. "Simply didn't capture the degradation users were reporting" says the signal existed in the world and not in the metric. "Relied too heavily on noisy evaluations" says the metric moved enough on its own that a real movement did not stand out.

Why a fixed suite can miss a routing fault

Take the context window routing error as the clean case. A routing fault affects the subset of requests that route a particular way. Every other request is fine.

Now think about what a fixed suite measures. It runs a golden dataset (a fixed set of labelled inputs with expected outputs, versioned so two runs can be compared) and reports one aggregate per scorer (a function that turns one output into a number or a verdict). If the affected path is hit by a small share of cases, the aggregate moves by a small amount. If the aggregate already wobbles by a similar amount run to run, the fault is inside the noise band and no threshold (the agreed number at which a score change is treated as a finding rather than as noise) fires.

Nothing about that is a bad dataset. It is arithmetic. A sparse fault plus a coarse aggregate equals a silent period, and the fix is not a better model, it is a different reading.

Reading a number with its variance attached

The practical response to "noisy evaluations" is to stop reporting a single figure as if it were exact.

Galtea recommends measuring a judge (an LLM asked to score another model's output against a written rubric) with accuracy, precision, recall, F1, Pearson correlation, Spearman correlation and Cohen's kappa rather than one statistic, and writes that a judge with 0.9 accuracy and 0.1 Cohen's kappa[src] is essentially guessing.[Galtea] Hamel Husain writes that raw agreement can be misleading when classes are imbalanced, and that human labels should be treated as ground truth with the judge true positive rate and true negative rate reported separately. His worked example is that a judge which always predicts Pass still scores 95 percent agreement when errors occur in 5 percent of examples.[Hamel Husain]

Both points land on the same habit. One headline number hides the direction of the error, and the direction is what tells you whether your detector is asleep. Our judge protocol requires more than one statistic with the label counts behind it, and the monthly report structure puts every number next to the dataset it came from.

Users had the signal before the metric did

The other thing the postmortem records is where the evidence actually was: in what people were reporting. A fixed dataset is a sample of the past. A production fault lives in the present.

Langfuse says observation-level evaluators are the recommended target for live production data, and lists sampling a percentage of traces as a way to manage the cost of judge-based evaluation.[Langfuse] That page publishes no recommended sampling percentage, so we do not state one either. Our production sampling protocol sets the rate per plan, with the monthly allowance written on the plans page.

Sampling live traffic changes what a sparse fault looks like. A routing-dependent fault that touches a small share of requests will appear in a large enough sample, and it will appear in the affected slice rather than in the average, provided you keep the slice. The dataset also has to keep moving. Future AGI recommends refreshing a golden dataset monthly by default, sampling 200-500 recent production traces[src] stratified by intent and persona, labelling them against the active rubric and versioning the dataset as a new artifact, with weekly refreshes during rapid change and immediately after model or corpus updates.[Future AGI]

We are not claiming we would have caught this. The stated causes were in Anthropic's serving infrastructure, which no customer-side evaluation reaches. We cannot see a provider's routing layer, its compiler or its hardware, and nothing on this site implies otherwise. Read what we cannot do for the boundary in full.

What a team should take from it

Three things, and none of them is about buying anything.

Know your noise band before you need it. Run your suite twice on the same configuration and write down how much the aggregate moves. A threshold set without that number is a guess, and it is the guess that produced the silent period here.

Keep slices, not just averages. Score by intent, by input length, by route, by tenant. A fault that touches one slice is invisible in the mean and obvious in the slice.

Treat user reports as a measurement. They arrived first in this incident. A support queue that is not read back into the dataset is a detector you own and do not use. Our chat assistant hub lists the scorers that apply once those reports become cases, and the eval baseline is where the first slice list gets written down.