Situations
Six things that send a team looking for evals. Each page says what is actually happening, what to do day by day this week, and which plan fits if you decide to hand it over.
These are written for the week you are in, not for a buying cycle. If the answer is that you do not need us, the page says so.
- We cannot decide between two modelsA public benchmark answers someone else's question. Build a small labelled dataset of your own cases, run both candidates on it, and check each lifecycle.5-step week . EV-DATA, EV-SCORE, EV-JUDGE
- An enterprise customer asked for evaluation evidenceThey want a method and a document, not a score. What you can hand over: a published protocol, a versioned dataset, a calibrated judge and a dated report.5-step week . EV-REPORT, EV-JUDGE, EV-DATA
- Our judge says pass and our users say noA judge that agrees with your people most of the time can still miss the failures that matter. How to recalibrate it against human labels this week.5-step week . EV-JUDGE, EV-DATA, EV-REPORT
- We are migrating from one provider to anotherDeprecation deadlines set the schedule, not you. Baseline the feature on the model you are leaving first, then read what the ported prompt actually changed.5-step week . EV-RERUN, EV-DATA, EV-SCORE
- A model update changed our outputsWhat to do when a provider update moves your feature: re-pin to a dated model id, rerun against the last passing run, then decide what to change.5-step week . EV-RERUN, EV-DATA, EV-THRESH
- A prompt edit regressed something two weeks laterThe edit shipped without a score and the dataset did not hold the broken case. How to find the edit, close the coverage gap, and gate the next one.5-step week . EV-DATA, EV-THRESH, EV-RERUN