What Is Max Tokens in AI, and What Does It Cap?
What is max tokens in AI? It caps output only, not input, and the stop reason tells you when a response was truncated rather than simply finished.
Max tokens in AI is the ceiling you set on how much the model may write in one response. It caps output only, it does not reserve space, and it has nothing to do with how much you can send in. When an answer stops mid-sentence, this setting is usually why.
Three different limits get called the same thing in conversation, which is the source of most of the confusion:
Limit | Who sets it | What it covers |
|---|---|---|
Context window | The model | Everything in one request: your input plus the response |
Maximum output tokens | The model | The most the model can produce in one response, whatever you ask for |
| You, per request | Your own ceiling on this response, up to the model's maximum |
What max tokens does in an AI request
Anthropic's Messages API reference describes max_tokens as "The maximum number of tokens to generate before stopping" and adds that "our models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate."
Two consequences follow from that phrasing, and both get misread.
It is a ceiling, not a target. Setting 4000 does not produce a longer answer. The model writes until it is finished or until it hits your number, whichever comes first. If you want longer output, ask for it in the prompt and make sure the ceiling is high enough not to interfere.
It is not a budget you pay for. You are billed for the tokens actually generated, so a high ceiling on a short answer costs the same as a low one. The reason not to set it enormous is different: it is a safety rail against a runaway response, and on some models it interacts with how much room is left for input.
The reference also notes that different models have different maximum values for the parameter, that the minimum is 0, and that setting it to 0 populates the prompt cache without generating a response. On models that produce extended thinking, it states that thinking tokens count toward the max_tokens limit, which is the detail that catches people out: a reasoning-heavy response can spend much of your ceiling before the visible answer starts. What that mode costs in general is covered in what reasoning effort is and what it costs.
How to tell a truncation from a short answer
This is the practical skill, and it does not require reading the text. The response tells you why it stopped.
Anthropic's stop reasons guide lists max_tokens as the case where the response reached the request's limit, alongside end_turn for a natural finish and stop_sequence for a sequence you supplied. The recommended handling for max_tokens is to raise the limit or continue the response.
resp = client.messages.create(model=..., max_tokens=1024, messages=[...])
if resp.stop_reason == "max_tokens":
# the answer is incomplete, do not store or show it as final
...Three things follow from checking that field:
Never render a truncated response as a finished one. A user reading a sentence that ends mid-word will assume the model is broken rather than that your ceiling is low.
Never write a truncated response into a database that expects complete records, especially JSON. A cut-off JSON object is invalid, and that failure surfaces far from its cause.
When using tools, a truncated response can end with an incomplete tool call, which the stop reasons guide specifically warns about. Checking the stop reason before parsing saves a confusing exception.
In a streamed response, the value arrives on the final message delta rather than at the start, so a streaming interface has to handle this at the end of the stream.
Choosing a number
Work from the longest legitimate answer rather than from a round figure.
Use | A reasonable ceiling | Why |
|---|---|---|
Classification, one label | 10 to 50 | Caps a model that starts explaining itself |
Short JSON object | 200 to 500 | Enough for the record, small enough to catch runaway output |
Email or summary | 500 to 1500 | A page of prose is roughly 500 to 800 tokens |
Long document or code file | 4000 and up | Match it to the longest real example you have |
A good default procedure: take your longest real output, count its tokens, and set the ceiling to roughly twice that. The doubling absorbs variation without permitting an unbounded response. If you do not know how many tokens your typical task produces, estimating tokens for a task gives you a method.
A low ceiling is also a legitimate tool for controlling cost and latency on a high-volume endpoint, since output tokens are usually the more expensive side of the bill. Just make it deliberate, and handle the truncation when it comes.
Where it sits next to the other limits
The context window is the whole request plus the response, so a long input leaves less room for output regardless of what you set. What a context window is covers that interaction. The unit being counted in all of these is explained in what a token is, and the other way to end a response early, by defining text that terminates it, is in what a stop sequence is. For how these parameters fit into the way models produce text at all, see how AI models work.
FAQ
Does a higher max_tokens cost more?
Not on its own. You pay for tokens generated, not tokens allowed. The ceiling only costs you when the model actually uses the room.
Why did my answer stop at exactly the same length twice?
That is the signature of hitting the ceiling rather than a model quirk. Check the stop reason, and if it is max_tokens, raise the limit.
Can I set max_tokens higher than the model's maximum output?
No. The reference states that different models have different maximum values for the parameter, and a request above a model's limit is rejected rather than silently reduced.
What should I do when a response is truncated?
Either raise the ceiling and retry, or ask the model to continue from where it stopped. For structured output, retrying with a higher ceiling is safer, because stitching two halves of a JSON object together is more fragile than paying for one more call.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
