Test the Compensations, Not the Happy Path

This is the test I would keep if I could keep only one:

it('restores the verification token and leaves no row when the platform create fails', function () {
    // … arrange a lead and a verified-email token …

    AgentServer::tool(CreateSiteTool::class, [/* … */]);

    // The single-use token is restored so the user can retry without
    // re-verifying, and the rolled-back provisioning row is not left behind.
    expect(Cache::get("verified-email:{$token}"))->not->toBeNull();
    expect(ProvisioningSite::query()->count())->toBe(0);
});

Two assertions, and between them they encode the whole of Chapter 10. The upstream fails; afterwards the user's verification still works and no orphan row is squatting on their URL.

Note what makes this test possible: the entire file fakes a 500 for every outbound request. The failure path deserves its own test file, with the failure arranged once in beforeEach, rather than one sad-path test hidden among twenty happy ones.

And note what it does not assert — the error message's wording. It asserts the state of the world afterwards, which is what the user experiences.

Contract Tests Beat Output Tests

Chapter 2 introduced the pair that binds prompt to schema, and they belong in any list of the highest-value tests here: assert the prompt declares every section the schema requires, and assert the resource exposes exactly the schema's union. No tokens, milliseconds to run, and they catch the specific drift that nothing else can see.

The general form: assert that two descriptions of the same contract still agree. Prompt and validator. Tool schema and tool validation. Enum in a tool and enum in the service behind it.

Record Real Cases as Fixtures

For the parts that are genuinely about judgement, tests are how you keep a decision from being re-litigated:

it('keeps the provider ordering when two candidates cover the query equally', function () {
    // From live Pixabay results for "mopping office floor": both a photo
    // tagged "mopping, floor" and a home-office photo tagged "office, floor"
    // cover two terms. Weighting terms by position preferred the home office;
    // provider relevance order is the better arbiter.
    $candidates = [
        ['url' => 'https://img/mopping.jpg', 'alt' => 'kid child boy mopping jeans wooden floor eyeglasses door'],
        ['url' => 'https://img/home-office.jpg', 'alt' => 'home office notebook couch sofa computer laptop floor'],
    ];
    // …
});

Real data, copied out of a real response, with a comment explaining what it demonstrates. That is a measurement written down — the subject of the next chapter — and it means the alternative that was tried and rejected cannot quietly come back.

What Not to Test

Not the wording of a prompt. It breaks on every legitimate edit and teaches people to delete tests.

Not the model's output. "The reply mentions the business name" passes on Tuesday and fails on Wednesday for no reason anyone can fix.

Not the provider's behaviour. Their contract is theirs. Test what you do with what they send, including the shapes you have never seen.

If you want confidence in output quality, that is a different instrument — a small set of recorded cases you re-run deliberately when you change a prompt, and read with your eyes. Useful, and not a CI gate.

What This Chapter Argued

• You cannot assert on the model, and it does not matter. Every bug in this book lived in the code around it.
• preventStrayRequests before Http::fake. Otherwise an unmatched request quietly reaches a paid API.
• Test the loop by feeding it stop reasons. Each branch is a case, and a new one from the provider becomes a one-line addition.
• Exercise tools through the server, so registration, schema and validation are covered — and assert that unpublished tools stay unpublished.
• Give the failure path its own file, with the failure arranged once, and assert the state of the world rather than the wording.
• Assert that two descriptions of one contract still agree.
• Turn measurements into fixtures so a rejected alternative cannot quietly return.
• Do not test prose, output, or the provider.