What a Token Costs You

Every other failure in this book announces itself. A malformed response throws, a timeout logs, a truncated reply looks wrong on screen.

Cost does not. Cost accumulates quietly, correctly, one perfectly successful request at a time, and tells you at the end of the month. By then the only remaining question is how large the number is.

That asymmetry is why spending is treated here as a correctness property rather than an operational concern. An AI feature without caps is not a cheap feature that might get expensive. It is an unfinished feature.

The Bill Has More Lines Than You Think

Before capping anything, know what you are being charged for. Four numbers come back on every call, and they behave differently:

• Input tokens — everything you sent. Prompt, conversation history, tool definitions, tool results. Grows with the length of a conversation, not with the length of the answer.
• Output tokens — what came back. Usually the smallest number and, on many providers, the most expensive per token.
• Cache reads — input tokens served from a cached prefix, billed at a fraction of the normal rate.
• Cache writes — the cost of putting a prefix into the cache, which is usually slightly more than sending it uncached. Caching pays back on the second call, not the first.

And then the multiplier nobody puts in the estimate: a turn is not a call. In an agent loop the whole conversation, including every tool result so far, is re-sent on every iteration. A turn that uses four tools does not cost four tool calls; it costs four increasingly large requests. Chapter 8 is about controlling that loop. This chapter is about making sure it cannot run away while you do.

The Cap Is a Server Decision, Made Per Purpose

The first rule of spending: the client does not get to choose anything that costs money. Not the model, not the token limit, not the provider.

This is easy to get wrong, because the natural shape of an API endpoint is to accept parameters. The natural shape is wrong here. A model parameter is a request for the most expensive model you support, sent by whoever reads your JavaScript.

The pattern that solves it cleanly is a FormRequest that overwrites the request before validating it:

protected function prepareForValidation()
{
    $isTranslate = $this->input('purpose') === 'translate';

    $this->merge([
        'api' => Config::string('services.ai.default'),
        'model' => $isTranslate
            ? Config::string('services.ai.drivers.openai.translate_model')
            : Config::string('services.ai.drivers.openai.prompt_model'),
        'max_tokens' => $isTranslate
            ? Config::integer('services.ai.drivers.openai.translate_max_tokens')
            : Config::integer('services.ai.drivers.openai.prompt_max_tokens'),
    ]);
}

prepareForValidation() runs before the rules do, so whatever the caller sent for api, model or max_tokens is discarded and replaced with a server-chosen value. The rules may still list those fields — they will never see a client's version of them.

The only knob the caller gets is purpose, and it is an enum:

'purpose' => ['sometimes', 'string', 'in:prompt,translate'],

That single word maps to a lane: a fixed pairing of model and output cap chosen for a kind of work. Rewriting one field on a page is a cheap model with a small cap. Translating a whole site is a more capable model with a larger one, because a truncated translation is a broken page.

Two things make this worth copying.

Lanes are the right granularity. Not one global cap, which is either too small for your largest job or too generous for your smallest. Not a free parameter, which is not a cap at all. A short list of named purposes, each with its numbers pinned server-side.

Adding a lane is a deliberate act. A new kind of work does not inherit a cap by accident; someone chooses its model and its ceiling, in a commit.