Skip to content
DriftCaught

Entry product

Eval baseline

The eval baseline is $1,800 one-time and takes 10 business days. You get a golden dataset built from your real failures, scorers, a calibrated judge, agreed thresholds and a first report.

$1,800one-time, credited to the first month of any plan started within 60 days

Any team with one AI feature in production and no eval suite it trusts.

Start Eval baselineRead the method first

What counts as a feature

A feature is one AI capability with its own prompt or pipeline and its own golden dataset. Two prompts behind one button are one feature. One prompt serving two products is one feature. A chat assistant and the summariser next to it are two.

This plan covers 1 feature.

Scope and limits

Scope and limits
Features covered1
Turnaround10 business days
Ongoing rerunsNot included, a monthly plan covers them
Walkthrough45 minutes

What you receive

  • Golden dataset built from your real failures and representative inputs, labelled and versioned
  • Scorers selected per feature type, judge calibrated against human verdicts with agreement reported
  • Thresholds agreed with you
  • First report with current scores and gaps
  • 45-minute walkthrough

What triggers a rerun

Nothing. This is bought once and produces one report. It has no ongoing triggers, and saying otherwise would be the easiest thing on this page to get wrong.

A monthly plan is what adds them: a tracked provider update, a prompt or retrieval change you flag, the weekly schedule, and a CI gate on the platform plan. The baseline hands you a working suite; keeping it current is a separate decision.

EV-RERUN, the triggers and the procedure in full

Not included

  • Reruns after the baseline
  • Production sampling
  • Monthly reports

Protocols this plan runs

Each protocol is published in full, with citations and a version. Nothing in the service happens by a method that is not on this site.

Getting started

  1. 01

    Access

    You give us sample inputs and outputs, the prompt or pipeline, and a way to run it. A staging key and an export of logged requests is usually enough.
  2. 02

    Dataset

    We pull your real failures out of those logs, label them with your domain expert, and version the set. This is the part nobody else wants to own.
  3. 03

    Scorers and judge

    We pick the scorers that fit your feature type, calibrate the judge against the human verdicts, and report the agreement so you know what the score is worth.
  4. 04

    Thresholds and handover

    You agree the thresholds that gate a release. From then on every trigger reruns the suite and you get a written verdict.

What we guarantee

  1. First eval baseline within 10 business days of access.
  2. Every tracked provider update triggers a rerun within 2 business days of the provider's announcement.
  3. Every flagged prompt or retrieval change is evaluated before it ships, within the plan's turnaround.
  4. Monthly drift report by the 5th business day.
  5. Judge recalibrated against human verdicts at least monthly.
  6. If a regression above the agreed threshold reaches production on a change we marked as passed, the next month is free.

These six are the whole list. Nothing else on this site is a guarantee.

What we cannot do

We cannot make a model deterministic or guarantee its outputs. We measure against an agreed dataset and thresholds and report what changed. We do not train or fine-tune models, and we do not replace human review for policy or legal decisions. Coverage is bounded by the dataset and the plan.

The longer version, with the reasoning

Exit terms

Monthly. Cancel with 30 days' notice. No setup fee beyond the baseline. The baseline fee is credited to the first month of any plan started within 60 days.

Questions about Eval baseline

What if we have no logged failures yet?
Then we build the first set from representative inputs you supply and from the cases your support queue already knows about, and we mark which items came from production and which did not. The set grows from real traffic once the feature is running.
Is the baseline fee really credited?
Yes. Start any plan within 60 days of the baseline and the $1,800 comes off your first month.
What do we get at the end of 10 business days?
A versioned golden dataset, the scorer set, a calibrated judge with its agreement figure, agreed thresholds, a first report, and a 45-minute walkthrough. The suite runs in your stack, so it stays with you whether or not you continue.
What counts as one feature?
A feature is one AI capability with its own prompt or pipeline and its own golden dataset. Two prompts behind one button are one feature. One prompt serving two products is one feature. A chat assistant and the summariser next to it are two.
Can I cancel?
Monthly. Cancel with 30 days' notice. No setup fee beyond the baseline. The baseline fee is credited to the first month of any plan started within 60 days.
Do you publish our results?
No. Your dataset, your scores and your reports are yours. The only thing this site publishes is the model change log, built from provider announcements, and the methodology. We publish no measurement of any model as our own.

Ready? The link opens an email with the four things we need to quote the work: feature type, provider and model, monthly volume, and whatever eval setup you have now. Start Eval baseline.

Other plans