Evaluation-driven development

0:00 Evaluation-driven development

In normal software, you write a test once and it passes or fails the same way forever. AI features aren't like that. The same input can produce different outputs. A prompt tweak can fix one case and break three others. A provider model update can shift behaviour overnight. The teams that ship reliable AI treat evaluation as the engine of development. In this lesson you'll learn the loop, the parts of an evaluation system, what product managers own, and how error analysis tells you exactly what to fix.

0:38 Analogy: the tasting spoon

Here's an analogy. Building AI without evaluations is like cooking for a restaurant without tasting. You might follow the recipe perfectly, but ingredients vary, ovens differ, and one day the supplier changes the flour. Tasting every batch is how chefs stay consistent. Your evaluation set is the tasting spoon, and you use it every time anything changes.

1:03 The loop

Here's the loop. Define success criteria from your AI brief. Build an evaluation set of real inputs with grading notes. Build the simplest version that could work. Run the evaluation, read the failures and categorise them. Change one thing: the prompt, retrieval, model, a tool or the UX. Re-run, and keep changes that improve results without regressions. Repeat until you meet launch criteria, and keep running after launch. It's test-driven development, adapted for probabilistic systems.

1:36 Five components

An evaluation system has five parts. Evaluation sets with representative, edge, failure and adversarial cases, tagged by category. Graders: code checks for objective criteria, rubric-based LLM judges calibrated against humans for subjective ones, and human review for samples. A harness that runs a version against the set and reports results by tag, compared with the previous version. Launch thresholds agreed in advance. And regression gates, so changes that drop key metrics below thresholds don't ship.

2:09 What product owns

Evaluation isn't only an engineering job. Product managers and domain experts should own what good means for users, which scenarios matter most, including high-risk ones, the expert grades that LLM judges are calibrated against, and trade-off calls, like whether a small quality gain is worth extra latency. If the people who understand the customer don't own the rubric, you'll optimise the wrong thing very efficiently.

2:37 Error analysis → online checks

Aggregate scores tell you how much. Reading failures tells you why. Take a sample of failures and categorise them: retrieved the wrong document, ignored a policy exception, wrong tone in Arabic replies, missing data in the source system. Fix the biggest category first. You'll often discover some failures aren't AI problems at all. They're data quality, unclear policy or confusing UX. Then confirm offline gains online: shadow mode, where the new version runs silently alongside, staged rollouts to a small share of traffic, and A/B tests with a primary metric, guardrails and duration decided before you start.

3:19 Simple example: a friendlier prompt

A simple example. You change one line in a prompt to make replies friendlier. Your evaluation set of sixty cases shows tone scores rise, but three refund-policy answers now leave out the thirty-day window. Without the set, you'd have shipped a friendlier assistant that gets policy wrong. With it, you add an instruction about policy details, re-run, and ship a version that's friendlier and still accurate.

3:48 Worked example: support drafts

Here's a worked example. A support-reply drafting feature meets its offline quality threshold and launches in pilot. Online, agents edit returns-related drafts far more than expected. Error analysis shows the drafts cite an outdated returns window, because a superseded document is still indexed. The fix is in retrieval, not the prompt, plus a regression case added to the evaluation set. Edit rates for returns fall the next week, and that error can never silently return.

4:21 Business example (illustrative)

Illustrative numbers for the support drafts. Offline, the pilot passed eighty-eight percent of a hundred and fifty cases. Online, returns drafts were edited in about sixty percent of cases versus twenty-five percent elsewhere. After removing the superseded document and adding three regression cases, returns edits fell to about thirty percent in a week, and the regression cases have passed on every release since.

4:48 Hands-on in the lesson

The hands-on section gives you a minimal harness you can run without buying a platform: cases in a JSON lines file, cheap code graders for must-include and must-not-match rules, an LLM judge that returns pass or fail with failed criteria, results saved per version, and a release gate that fails the build if any tag drops more than a few points or safety falls below its floor. Calibrate the judge against fifty human-graded outputs first, and record the judge's model and prompt version with every result.

5:25 Common mistakes

Common mistakes. Building the evaluation set after launch. Using only easy, typical cases. Trusting an LLM judge without checking it against human grades. Changing several things at once. Letting the evaluation set go stale while real usage evolves. And keeping evaluation in an engineer's laptop, so product and domain experts never see the results or influence the rubric.

5:50 How you'll know it's working

How will you know evaluation-driven development is working? Every change has a before-and-after result on record. Regressions are caught by the gate, not by customers. The share of failures that surprise you in production shrinks over time, because each one becomes a new test case. And product conversations shift from I think it's better to here's what changed, by tag.

6:16 Watch me do it: harness + gate

Watch me do it. I open the harness folder. Cases dot JSON L has one case per line with an ID, input, tags and expectations. In graders, must include checks required phrases, must not include checks forbidden patterns, and judge sends the rubric, input and output to a pinned judge model and parses pass or fail, treating unparseable output as a fail. In run dot py, evaluate runs my system on every case, applies code checks first, calls the judge only if a rubric exists, and saves results per version. By tag computes pass rates. Gate compares the new version with the baseline and fails if any tag drops more than three points or safety falls below one hundred percent. I run version eight against version seven. The output says returns, point nine one to point eight four, and exits with a failure, so the change doesn't ship until I fix it.

7:22 Recap

To recap: AI features are probabilistic, so evaluation is the core development loop. Build tagged sets, calibrated graders, a harness, thresholds and gates. Product owns what good means. Error analysis finds the biggest failure categories, and staged rollouts confirm offline gains. Your next step is to categorise thirty failures from a feature or prototype, estimate the fix for each category, and start with the biggest. For more depth, see Evaluating and Monitoring LLM Applications.

7:54 Try this now (45 minutes)

Try this now. Create a folder with a cases file and write fifteen cases for one AI feature: ten typical, three edge cases and two that should be refused or escalated. For each, write what must be included and what must not appear. Run your current version, record pass or fail, and categorise every failure. That's your baseline and your first improvement backlog.