In development your AI feature answers in three seconds and you conclude it is fast.
It is not fast. It was small. A short prompt, no history, no tools, one round trip. In production the same feature carries a full conversation, calls three tools, and takes ninety seconds — and ninety seconds is not an edge case, it is Tuesday afternoon.
Everything in this chapter follows from accepting that number instead of arguing with it.
The Web Request Is the Wrong Place
Run that turn inside the HTTP request and four things happen, in this order.
A PHP worker is held for the entire ninety seconds. Your pool is not large; a handful of concurrent conversations and the site stops answering any request, including the ones that have nothing to do with AI.
Something upstream gives up first. A load balancer, a CDN, a proxy — each with its own timeout, none of which you set, all shorter than you would like.
The user sees nothing. Not an error, not a partial answer. A spinner, then a blank failure.
And you paid for all of it. The model completed its work, the tokens are billed, and the result went nowhere because the connection that was supposed to carry it had already closed.
That last one is what makes this different from an ordinary slow endpoint. A slow database query that times out costs you a query. A model turn that times out costs you the whole turn, and you have nothing to show for it.
Move the Turn to a Queue
The shape is: the request dispatches a job and returns immediately; the job does the work and publishes its state somewhere the front end can read; the front end polls until the state says it is finished.
class ProcessAiMessage implements ShouldQueue
{
use Queueable;
public int $timeout = 120;
public int $tries = 1;
public function __construct(
public readonly string $cacheKey,
public readonly array $conversationHistory,
public readonly array $messages,
// ...
public readonly string $systemPrompt,
) {}
}
Two properties on that class carry most of the design.
$timeout = 120 is the outer bound on the whole turn. Not one call — everything.
$tries = 1. No automatic retry, ever. A turn that has already run has spent money and may have called tools with real side effects. Laravel's default of retrying a failed job is exactly wrong here: it would re-run a partially-completed agent turn, pay for it again, and possibly perform the same irreversible action twice. Retries in this world are a decision made per-tool, with idempotency behind them — which is Chapter 10, and it is why this number is 1.