Your AI API Bill Just Spiked: What to Check First

Cap the spend first, diagnose second. If your AI API bill has jumped and you do not yet know why, the correct first action is to set a hard spending limit at a number you can survive, because every minute you spend investigating is a minute the meter keeps running. Diagnosis is thirty minutes of work. An uncapped runaway loop over a weekend is a different kind of story.

Once the ceiling is in place, the cause is almost always one of four things, and each leaves a different signature in your usage graph.

Read the shape of the spike

Pull up your provider's usage view at hourly granularity, not daily. Daily granularity hides the difference between a step change and a spike, and that difference is the diagnosis.

The third row is the one that catches teams out most often. Cost per request is set by the model and the prompt, and both change silently. A model upgrade in a config file, a system prompt that grew by four paragraphs, or a retrieval step that started returning ten documents instead of three all produce the same flat step with no traffic change behind it.

The five checks, cheapest first

Work down this list. Most bills are explained by the first three.

1. Did request volume change, or did cost per request? Divide spend by request count for the day before and the day after. If the ratio moved, it is a prompt or model problem and traffic is innocent. If the ratio held, you have more traffic and the question moves to whether it is legitimate.

2. Did anything deploy? Check the timestamp of the step against your deploy log. Model swaps are the usual culprit, and a reasoning effort setting left on high is a very expensive one line change.

3. Is anything retrying? A failing call that retries three times costs four times as much and shows up as errors, not as usage. This is the single most common invisible multiplier. If your error rate rose alongside spend, fix the errors and the bill fixes itself.

4. Is your input growing? Conversation history, retrieved documents and appended context all inflate input tokens over time without anyone deciding to. A chat feature with no history trimming gets more expensive per message the longer a session runs. Sanity check by hand: multiply your average input length by your request count and see whether the number you get is anywhere near the one you were billed.

5. Is someone else using your key? Check for traffic from unexpected regions or times, and for calls to endpoints your app does not use. A key committed to a public repository gets found in minutes.

Stop the bleeding while you look

Four controls, in the order they take effect:

• Provider spend limit. Every major provider has one. Set it now at a number that hurts but does not kill you. This is the only control that works when your own code is the problem.

• Rate limit per user. Cap requests per user per hour at whatever your heaviest legitimate customer does, times two. This bounds one account taking the whole budget.

• Max tokens per request. An output cap on every call. It costs you nothing on well-behaved requests and truncates the pathological ones.

• A kill switch for background jobs. One flag that stops all scheduled AI work without a deploy. Most runaway bills come from something running unattended.

If you are running agents rather than single calls, the exposure is larger and the controls need to be tighter. Setting spending limits for AI agents covers the per-run budget pattern, and API rate limits covers what the provider will and will not stop for you.

After you find it

Two things worth doing in the same week, while the pain is fresh and you will actually do them.

Set an alert, not a limit, at the level you expect. Limits stop disasters. Alerts catch drift. Put the alert at roughly 130% of a normal day so you hear about the next one within hours rather than at the end of the month.

Write down your cost per unit of value. Cost per conversation, per document processed, per user per month. A single number. Without it you cannot tell whether next month's higher bill is a problem or a sign that things are going well, and the difference matters enormously. Reducing AI API costs has the levers once you know which unit you are optimising.

If a client is paying for this

A spike changes a conversation you are already having. Two rules that have saved a lot of relationships.

Tell them before the invoice does. A bill that arrives unexplained reads as incompetence, and the same bill explained in advance reads as attentiveness. Explaining AI costs to a client covers the framing.

Then decide whether your pricing structure is the actual problem. If you charge a flat fee and your costs are variable, you carry all the volatility, and one heavy customer can make a profitable contract unprofitable. That is a pricing design question, not a billing accident, and pricing an AI product with variable costs is where to take it. The wider set of models sits in how AI products actually make money.

FAQ

Why did my AI API bill go up without more users? Almost always cost per request rather than request count. A model change, a longer system prompt, more retrieved context, or retries on failed calls will all raise spend with flat traffic.

Can I get a refund for a runaway API bill? Sometimes, for clear-cut cases like a leaked key, and only if you ask promptly with specifics. Providers are not obliged to, so treat it as a possible outcome rather than a plan.

What is a reasonable spending limit to set? Start at roughly three times your normal monthly spend. High enough that legitimate growth does not trip it, low enough that a loop cannot run for a week before anyone notices.

How do I stop retries from multiplying my costs? Cap retry attempts, use exponential backoff, and do not retry on errors that will not succeed on a second attempt, such as validation failures. Log retries separately from first attempts so they are visible in your metrics.

Should I switch to a cheaper model after a spike? Only after you know the cause. Switching models to fix a retry loop or a leaked key just makes the same problem cheaper per unit, which is not the same as fixing it.