Why Are AI Models Bad at Counting?

AI models are bad at counting because they never see the thing you asked them to count. They see tokens, which are fragments of text of varying size, and counting requires holding a running total across those fragments with no place to put it. The model has no scratch memory between one predicted token and the next, so a count has to be reconstructed from scratch every time, from a representation that was never designed to preserve quantity.

This is why a model that can write a working sorting algorithm will still miscount the letters in a word. Those are not the same skill, and the second one is genuinely harder for the architecture.

What the model actually receives

Take the word "strawberry" and the question of how many times the letter R appears.

A tokeniser does not hand the model ten letters. It hands it something closer to three chunks, along the lines of str, aw, berry. Each chunk becomes a single vector. The letters inside a chunk are not separate items the model can iterate over. They are a property of an embedding, in the way that "reddish" is a property of a colour rather than a list you can walk.

So the question "how many Rs" arrives as: given these three vectors, what number follows? The model answers from pattern, not from procedure. It has seen a great deal of text about spelling and can often produce the right answer. It has no mechanism that touches each R in turn.

What a token in AI actually is covers the tokenisation step in more depth, and it is worth understanding because it explains a whole family of odd failures, not just this one.

Why there is no running total

The second half of the problem is architectural.

A transformer produces one token at a time. Each pass computes attention over everything so far and emits the next token. There is no register that persists across passes, no variable being incremented. The only memory between steps is the text already generated.

Which points directly at the fix, and explains why it works. If the model has nowhere internal to keep a count, give it somewhere external: the output itself.

Ask for a total and you get a guess. Ask it to list the items with an index and then state the total, and the intermediate steps become tokens in the context window, which the model can attend to on the next pass. You have turned an unavailable internal state into visible state. That is the whole mechanism behind chain of thought prompting, and counting is where the effect is easiest to see, because the difference between right and wrong is a single unambiguous number.

The failures you will actually hit

Letter counting is the famous one and the least important. Here are the ones that cost real work:

The pattern across all five: never ask for a number as the answer. Ask for the enumeration and take the number from it, or move the counting into code.

Why bigger models did not fix this

Larger models are better at this, up to a point, and the reason is unflattering. They have absorbed more text containing counts, so their approximation is closer more often. The mechanism did not change. A better guesser is still guessing.

You can see this in how the errors behave. A model that miscounts does not fail randomly. It tends to be off by one or two, and it tends to fail more as the collection grows, which is exactly the signature of an estimate rather than a broken procedure. Reasoning models help more, because they generate intermediate steps by default and those steps do the externalising described above. That is what reasoning effort in AI is buying you: not a different architecture, more room to write the working out.

The clean solution is different in kind: give the model a tool. A model that can call a function does not need to count, it needs to decide to call len(). That shifts the problem from an unreliable capability to a reliable one, which is largely what tool calling in AI agents exists for.

A quick way to test your own case

If you are unsure whether a counting step in your pipeline is safe:

1. Take twenty real inputs where you know the correct answer.

2. Run your prompt as it is and record accuracy.

3. Run it again asking for an enumerated list plus the total, and score only the total.

4. Compare.

If step three is much better, your prompt was asking for a number where it should have been asking for a list. If both are poor, the counting should not be in the model at all. Twenty examples is enough to tell those two situations apart, which is the only decision you need to make.

The general lesson

Counting is a small window onto a bigger property: a language model is a very strong pattern engine with no built-in procedural execution. Anything requiring a maintained state that is iterated over, whether a count, a sum, a position, or a running comparison, has to be made visible in the output or handed to code.

Once you have that framing, the failures stop being surprising. They are all the same failure, and how AI models work makes it fairly predictable which tasks will land on the wrong side of the line.

FAQ

Why can't AI count the letters in a word?

The model never sees individual letters. Text is split into tokens, and a word like "strawberry" becomes a few chunks whose internal letters are properties of an embedding rather than items that can be iterated. Counting them requires a step the architecture does not perform.

Does asking the model to think step by step fix counting?

Usually, yes. Writing the intermediate steps puts them in the context window where the model can attend to them, which substitutes for the internal counter it does not have. Ask for an enumerated list and then the total, rather than the total alone.

Are newer models better at counting?

Somewhat, because they approximate from more training data, and reasoning models more so because they externalise steps by default. Neither changes the mechanism, so both still fail on long collections.

Why can't the model produce exactly 100 words?

Generation decides one token at a time with no observable running word count. A range works better, or count the words in code afterwards and trim.

What is the reliable fix for counting in a production pipeline?

Move it out of the model. Give the model a tool that counts, or have it emit structured output that your code counts. Both replace an unreliable capability with a deterministic one.