A cap set carelessly causes a subtler failure than a large bill.
On models that reason before answering, the output limit covers both the reasoning and the visible reply. A limit that looks generous against the answers you expect can be consumed almost entirely by thinking, leaving the visible response to stop mid-sentence.
That is exactly what happened in the chat feature. The limit had been set when the model did not think before answering, and it was ample. After a model change, replies started ending mid-thought — and, because nothing checked why generation stopped, they rendered as though they were complete.
Two fixes, and you need both. Raise the cap to account for reasoning — and check the stop reason, so that when the cap is hit the user is told rather than shown a confident fragment:
if ($stopReason === 'max_tokens') {
report(new \RuntimeException('Response truncated on max_tokens'));
$state['messages'][] = [
'role' => 'bot',
'content' => __('messages.errors.ai_truncated'),
'is_error' => true,
];
$state['hasError'] = true;
break;
}
A cap you cannot observe hitting is a cap that silently degrades your product. Chapter 8 covers the full set of stop reasons; this one belongs here, because it is the direct consequence of a number you chose.
Two Limits, Not One
Rate limiting and budgeting look similar and answer completely different questions.
A rate limit answers: is this too fast? It protects the provider's quota, your workers, and the queue. It resets constantly — per minute, per few seconds — and it belongs on the route, where Laravel already does it well.
A budget answers: is this too much? It protects money. It resets daily or monthly, and it belongs in the handler, where you know which tenant is paying.
You need both, and neither substitutes for the other. A per-minute limit permits an enormous, evenly-paced monthly bill. A daily ceiling does nothing about a burst that saturates every worker in ten seconds.
Increment First, Ask Afterwards
Here is the budget, and the ordering is the whole point:
$perDay = config('site.ai_editable.per_day');
$cacheKey = 'ai-editable:'.$site->id.':'.now()->format('Y-m-d');
Cache::add($cacheKey, 0, now()->endOfDay());
$used = (int) Cache::increment($cacheKey);
if ($used > $perDay) {
return new ErrorResponse(
'Daily AI edit limit reached for this site.',
Response::HTTP_TOO_MANY_REQUESTS,
['Retry-After' => (string) now()->diffInSeconds(now()->endOfDay())],
);
}
The tempting version reads the counter, compares it, and increments if there is room. That version leaks. Under concurrency, twenty requests all read the same value, all decide there is room, and all proceed — and the one thing a spending limit must never do is fail open.
Incrementing first makes the counter authoritative. Every request that gets this far has already been counted; the comparison happens against a number that includes itself. Two requests racing at the boundary see different values and exactly one is rejected. The cost is that a rejected request still consumed a slot from the counter — which is the right direction to be wrong in.
Cache::add before increment seeds the key only if it is missing, in one atomic operation, and sets the expiry so the counter disappears at midnight instead of being reset by code that has to remember to run.