Why AI Models Struggle to Count Letters

AI models struggle to count letters in a word because they never actually see individual letters. Before a model reads your text, a tokenizer breaks it into chunks called tokens, and those chunks are usually sub-word pieces rather than single characters. Ask a model how many letters are in "strawberry," and it isn't scanning s-t-r-a-w-b-e-r-r-y one character at a time. It's working with two or three token pieces, and it has to infer the letter count from patterns learned during training rather than from a direct look at each character. That's the root cause, and it's a mechanical limitation of how the model reads text, not a sign of low intelligence.

The tokenizer is the bottleneck

Every request you send to an AI model passes through a tokenizer first. If you haven't seen what a tokenizer actually does, the short version is that it converts your text into a sequence of numeric IDs the model can process, and each ID usually stands for a chunk of a word, not one letter. Common short words often get their own single token. Longer or less common words get split into pieces, and where the split happens depends entirely on which sequences of characters showed up often enough in the tokenizer's training data to earn their own slot in its vocabulary. Nobody hand-designs these splits for letter counting. They're optimized to compress language efficiently, and letter-level accuracy was never part of the design goal.

A worked example: breaking down "strawberry"

Run "strawberry" through a typical byte-pair-encoding tokenizer, the kind most large language models use, and it doesn't come out as ten separate letters. It comes out as a small number of sub-word chunks, commonly something like "straw" and "berry," or in some tokenizers three pieces such as "str," "aw," and "berry." The exact split varies by model and tokenizer version, so don't treat any single breakdown as universal. What matters is the shape of the problem: the model's internal representation of the word is a short sequence of two or three token IDs, not ten individual letter IDs.

There is no step in between where the model decomposes a token like "berry" back into b-e-r-r-y before counting. Each token gets mapped to a vector, and the model reasons over those vectors. To answer "how many r's are in strawberry," it has to reconstruct the internal spelling of a token it never actually decomposed, relying on patterns it picked up from seeing that token used in context millions of times. That reconstruction is approximate. It's why the model can often get short, common words right while stumbling on longer or less frequent ones where the token boundaries hide more of the letter-level structure.

Why repeated letters are the hardest case

Double letters make this worse. In "strawberry," two of the three r's sit inside a single token the model treats as one indivisible unit, not as a visible run of r-r-y. Counting repeated characters requires noticing a pattern inside a chunk the model was never asked to look inside of. That's a fundamentally different task than counting tokens, which the model does easily and accurately, because tokens are exactly the units it was built to track.

This is a different problem than inconsistent answers

It's worth being precise here, because two separate AI quirks get lumped together a lot. The letter-counting problem is about tokenization, the model literally doesn't have direct access to individual characters inside a multi-character token. That's a completely different mechanism from why the same AI model can give you different answers to the same question asked twice. That inconsistency comes from randomness in how the model samples its next word, plus hardware-level floating point effects, not from anything related to how words get split into tokens. You can ask a model to count letters in the exact same word ten times and get inconsistent wrong answers for both reasons at once, but they're independent failure modes with independent fixes. Fixing one doesn't touch the other.

The transformer underneath never sees characters either

It helps to zoom out one level. The tokenizer hands its output to a transformer, the neural network architecture doing the actual reasoning. A transformer model processes a sequence of token vectors and learns relationships between them using attention, but it has no separate mechanism for looking inside a token at the character level. Characters simply aren't a unit the architecture operates on downstream of tokenization. So even a much larger, more capable transformer built on the same tokenizer inherits the same blind spot for exact letter positions, because the limitation is introduced before the transformer ever runs, not by a lack of model capacity.

What actually helps when you need an exact count

If you need a reliable letter count, don't ask the model to eyeball it. Ask it to first spell the word out with a hyphen or space between every letter, like s-t-r-a-w-b-e-r-r-y. That step forces the tokenizer to represent each character as its own distinct piece in the visible context, which gives the model something closer to real per-letter access before it counts. It's a workaround, not a fix, since the underlying tokenizer hasn't changed, but it noticeably improves accuracy on this specific kind of task. Letter counting is one instance of a broader pattern covered in more general terms in why AI models are bad at counting, and it connects back to the fundamentals in how AI models work if you want the fuller picture of what happens between your prompt and the model's answer.