Running an eval suite in CI: what a gate does
A gate on every pull request changes review, deploy timing and who argues with whom. Here is the practical shape of it, and the class of change it cannot catch.
Tooling
Fair comparisons of the eval tools teams actually use, and the honest line between buying a tool and buying someone to own the evaluation.
Honest comparisons of eval tools, and where a tool stops and a service starts.
4 posts
Eval tool or eval service: which one you need
If someone on your team owns the dataset, the thresholds and the judge calibration and has time for them every month, buy a tool. The service exists for when nobody does.
Promptfoo vs DeepEval: choosing a starting point
Promptfoo's docs start at the build that fails. DeepEval's start at the dataset that is too small. Both are reasonable places to begin and the choice is reversible.
Braintrust vs Langfuse: what the docs say
A comparison built only on what each vendor documents, with the honest conclusion that for most teams either one works and the bottleneck is elsewhere.