There are three clocks and they must be nested, or the outermost one kills you before the inner ones can do their jobs.
The HTTP timeout on each individual call to the provider. The job timeout, after which the queue worker kills the job outright. A wall-clock deadline you enforce yourself, set inside the job timeout:
// Stop starting new round-trips once we're close to the job timeout, so
// handle() always reaches the Cache::put below instead of being killed
// mid-turn and leaving the widget polling a key that never appears.
$deadline = now()->addSeconds($this->timeout - 20);
The twenty-second margin is the whole idea. When the worker kills a job at its timeout, it kills it wherever it is — mid-loop, before it has written anything. The front end is left polling a cache key that will never appear, and the conversation simply stops talking. No error, no message, nothing.
By setting your own deadline short of the real one, you guarantee the job always reaches its final write. The margin is not for the model; it is for you, to finish tidying up.
And the per-call timeout is derived from what is left, never set as a constant:
$remaining = (int) max(0, now()->diffInSeconds($deadline, false));
if ($remaining < 5) {
report(new \RuntimeException('Skipped call — job budget exhausted'));
return null;
}
$response = Http::connectTimeout(10)
->timeout(min(60, $remaining))
// ...
min(60, $remaining) — a normal call gets sixty seconds, and a call made late in the turn gets only what remains. Without that, three sixty-second attempts plus backoff comfortably exceed a hundred-and-twenty-second job and get it killed before it can save anything.
Never start work you cannot finish. If under five seconds remain, do not make the call at all. Paying for a response that arrives after you have been killed is the worst of both outcomes.
Publish as You Go
The naive loop accumulates messages and writes them at the end. It works, and it makes a tool-using turn look broken, because the interface sits silent for a minute while three tools run.
Publish after every iteration instead:
// A tool-using turn can take several round-trips, and holding every message
// back until the loop ends made the widget sit silent for the whole turn.
$this->publish($state, $history, done: false);
private function publish(array $state, array $history, bool $done): void
{
$state['conversationHistory'] = $history;
$state['done'] = $done;
Cache::put($this->cacheKey, $state, now()->addMinutes(10));
}
The done flag is the contract with the front end: keep polling, or stop. Without it the client has to infer completion from the content, which means guessing, which means either a poll that never stops or one that stops early.
This is not streaming. It is coarse — a write per round trip rather than per token — and for a tool-driven feature it delivers most of the benefit for a fraction of the complexity. The user sees each step land as it happens.