Every provider publishes a price per million tokens. Almost nobody's bill matches a naive calculation from that number, because three multipliers sit between the sticker price and the invoice.
Multiplier one: the context you resend
In a conversation, the entire history is re-sent on every turn. A twenty-turn chat does not cost twenty messages — it costs roughly the square of that growth. Prompt caching, when the provider offers it, is usually the single highest-leverage optimisation available.
Multiplier two: output tokens
Output is typically priced three to eight times higher than input. Reasoning modes make this worse: hidden thinking tokens are billed as output even though you never see them. Constrain response length explicitly rather than hoping the model is brief.
Multiplier three: retries and failures
Malformed JSON, refused requests and timeouts are all billed. A pipeline with a fifteen percent retry rate is paying a fifteen percent tax on top of everything else.
Practical reductions
- Route by difficulty: send classification and extraction to a small fast model.
- Cache aggressively — system prompts and documents rarely change between calls.
- Use structured output modes to eliminate parse-failure retries.
- Trim conversation history to a rolling window plus a summary.
- Set hard max-token ceilings on every endpoint.
Teams that apply all five routinely cut spend by sixty to eighty percent with no measurable quality loss.
The aiz.to editorial team
We buy every subscription at retail price, run the same task suite across providers, and publish the scores unedited. Read the full methodology on our about page.
How we test