Required Paths Are a Contract — Keep Them Small
With one schema in place, validation becomes a loop:
public function __invoke(array $content, string $category): void
{
foreach (ContentSchema::requiredPaths($category) as $path) {
$value = self::get($content, $path);
if ($value === null || $value === '' || $value === []) {
throw new RuntimeException("AI site content response missing required key {$path}.");
}
}
}
Now look at what requiredPaths() actually returns:
public static function requiredPaths(string $category): array
{
return ['hero.title'];
}
One path. After all that, one.
This is deliberate, and it is the part people get wrong when they discover schema validation and immediately require forty keys. Required means the caller should throw the response away and try again. That bar is much higher than "would be nice to have".
hero.title is on the list because its absence commits a broken file to a repository. brand is not on the list, despite being essential, because the caller backfills it from the request inputs — the model's version is not the source of truth, so its absence is not a failure. A missing testimonial is not on the list either: three testimonials instead of the requested three-and-a-picture is a slightly worse site, not a corrupt one.
Every path you add to that list is a retry you will pay for, and a failure mode you have chosen to create. Add the ones where the alternative is corruption. Leave the rest.
Constrain the Model Away From What It Cannot Know
A structurally perfect response can still be a liability, and this is the failure with the longest reach, because nothing in your stack can detect it.
We generate an FAQ section for each site. Early on the instruction was, in effect, "write six frequently asked questions and answer them." The model obliged. The JSON was immaculate.
Real sites went live quoting a price range the business had never set, and describing a twenty-four-hour cancellation policy the owner had never agreed to.
The model was not malfunctioning. It was asked to write an FAQ for a business, and businesses have prices and cancellation policies, so it wrote plausible ones. Every schema check in the world passes that content. The keys are present, the strings are non-empty, the grammar is better than mine.
The fix is not validation. It is in the prompt, and it has two halves — steer away from the topics, and give it something to do instead:
Generate a frequently asked question related to this business, covering what to
expect, how the service works, how to get in touch or book, preparation, or the
area served. Do NOT ask about prices, discounts, or refunds.
Provide an answer using ONLY the facts given in the profile above. If a specific
figure is not supplied, invite the customer to contact the business instead of
stating one.
That second sentence is the load-bearing one. "Do not invent a price" leaves the model with a question it must still answer, and it will. "Invite them to contact the business instead" gives it a correct move. A prohibition without an alternative is a prompt the model has to disobey to finish the job.
The general rule: any field where the model can produce a specific, checkable, external fact — a price, a date, a phone number, a policy, a statistic, a person's name — is a field where it will eventually invent one, and no downstream code can tell. Either supply the fact, or instruct it to defer.