What Is Test-Time Compute in AI?

Test-time compute is the compute a model spends after you hit enter, while it is producing the answer, as opposed to the compute spent months earlier during training. In practice this mostly shows up as reasoning tokens: the model writes out intermediate steps, checks its own arithmetic or logic, sometimes discards an approach and starts over, before it commits to the response you actually see. Give a model more of this inference-time budget and it gets measurably better at math, code, and multi-step planning. Take the budget away and the same underlying weights answer faster and worse. That is also the whole reason reasoning models feel slower than a plain chat model doing the same job: they are spending more work per question, not carrying more parameters.

What "thinking longer" actually looks like

A reasoning model does not get bigger when you ask it a harder question. It runs the same trained weights every time. What changes is how many tokens it generates before the part you read. Ask a model at low reasoning effort to solve a logic puzzle and it might produce a few dozen tokens of scratch work before answering. Ask the same model at high reasoning effort and it might produce a couple of thousand: working through cases, checking a calculation, dropping a wrong approach, and trying another. Those extra tokens are the test-time compute. They are generated on the same hardware, at the same cost per token, as any other output, so more of them directly means a slower response and a bigger bill for that one query.

The trade-off, in plain economics

Every additional reasoning token is inference you pay for once, on that specific request, whether or not it changes the final answer. On an easy question, a model that reasons for a few thousand tokens before replying is usually wasting time and money, since a direct answer would have come out identical. On a genuinely hard question, those same tokens can be the difference between a wrong answer and a right one, because the model gets room to explore, verify, and correct itself before committing. Using these models well means matching reasoning effort to how hard the task actually is, not defaulting to maximum thinking on everything, which is expensive by design and slow by design.

A live example: Gemini 3.7 Flash reasoning harder, not bigger

Google's Gemini 3.7 Flash, released in August 2026, is a clean illustration of test-time compute paying off inside the same model tier. Google credits the jump largely to the model reasoning more diligently through multi-step planning and tool calls, not a bigger base model. Two results stand out:

• FrontierCode 1.1 Main, a coding benchmark: 34.4 percent for the prior 3.6 Flash, up to 43.6 percent for 3.7 Flash.

• DeepSWE v1.1, a long-horizon software engineering benchmark that measures multi-step, multi-file tasks: 49.0 percent up to 65.3 percent, the largest single gain anywhere in the model card.

Same weight class, more disciplined reasoning per task, meaningfully better results, and a slower, costlier answer to get there. That is test-time compute working exactly as designed.

Test-time compute versus a bigger model

It helps to keep the two levers separate, because they solve different problems. Training compute buys a smarter model in general: more parameters, more data, more floating point operations spent once, before release, baked permanently into the weights. Test-time compute buys a smarter answer to one specific question, spent every time that question is asked, and gone the moment the response finishes. A smaller model given a large reasoning budget can beat a much larger model answering in one shot on tasks that reward careful, step-by-step work, like a proof or a multi-file refactor. That same smaller model given no reasoning budget will lose to the larger model on almost everything else. Neither lever replaces the other. Frontier labs are stacking both at once.

Why the hardware race is about making reasoning cheaper

Because test-time compute is spent on every single request, it scales with usage in a way training cost never has to. That is the economic problem behind OpenAI's new Jalapeno inference chip, built with Broadcom and announced in August 2026. On the InferenceX benchmark, Jalapeno delivered more tokens per user and more throughput per kilowatt than a comparable Nvidia Blackwell system, largely by keeping model state local and cutting the data movement that slows down long reasoning traces. Deployment starts in small volumes in late 2026. None of this changes what test-time compute is. It changes what it costs, and at a lab's scale, inference already dwarfs training spend, so shaving cost off every reasoning token compounds fast.

Where this fits next to "what is a reasoning model"

This piece is about the mechanism: why generating more reasoning tokens at answer time trades latency and cost for accuracy, and how that differs from training a bigger model in the first place. The companion question, when it is actually worth paying for that trade, is covered in what is a reasoning model, which walks through the cases where extra thinking earns its cost and the cases where it just burns money on a question a fast model would have answered correctly anyway. For the deeper mechanics of how a prompt becomes an answer at all, including where reasoning tokens sit in that pipeline, see how AI models work. And if you want the knob itself, the setting that controls how much a model reasons before answering, see what is reasoning effort in AI.

Frequently asked questions

Is test-time compute the same as inference time compute?

Yes. Test-time compute and inference-time compute describe the same thing: compute spent while the model is answering a live request, as opposed to compute spent during training. Test-time compute is the more common phrase in reasoning-model papers, inference-time compute shows up more in infrastructure and cost discussions, but they point at the same tokens.

Why are reasoning models slower than regular chat models?

A reasoning model generates a chain of intermediate reasoning tokens before its final answer, and every one of those tokens takes real time to produce, one after another, on the same hardware as any other output. A regular chat model skips straight to the answer. More reasoning tokens means more sequential generation steps, which is exactly why reasoning takes longer, not because the model itself is bigger or slower per token.

Does more test-time compute always mean a better answer?

No. On tasks with a single obvious answer, extra reasoning tokens mostly restate the same conclusion in more words, adding cost and latency without adding accuracy. The gains show up on tasks with real branching: math proofs, multi-file code changes, planning problems where an early wrong turn compounds. Matching reasoning effort to task difficulty is the actual skill, not maximizing it by default.

Can a small model with more test-time compute beat a bigger model?

On specific task types, yes. A smaller model given a generous reasoning budget can outperform a larger model that answers in one pass, particularly on problems that reward verification and backtracking. It will not generally beat the larger model on broad knowledge or tasks that do not benefit from step-by-step reasoning, since no amount of test-time compute adds information the model was never trained on.

Why is OpenAI building its own inference chip instead of just using more GPUs?

Because test-time compute is spent continuously, on every request, inference cost scales with usage rather than being a one-time expense like training. OpenAI's Jalapeno chip, built with Broadcom, is aimed specifically at cutting that per-token inference cost and latency at the scale a reasoning-heavy product generates, rather than at training larger models faster.