What Is a Frequency Penalty in AI? A Worked Example
What is a frequency penalty in AI? See how it lowers the score of repeated words, how it differs from presence penalty, and which value to try first.
What is a frequency penalty in AI? It is a setting that lowers the score of a word every time that word has already appeared in the answer, and the more often it has appeared, the bigger the drop. The result is text that is less likely to loop, stutter, or reuse the same phrase over and over. It does nothing to the first use of a word, which is why it changes style rather than meaning.
If you have ever watched a model write "the best way to improve is to improve your improvement," you have seen what this setting exists to fix.
How a frequency penalty works, in plain terms
A language model does not pick words directly. At each step it scores every possible next token, then turns those scores into probabilities and samples one. Those raw scores are called logits.
A frequency penalty edits the logits before sampling. For each token, it subtracts a small amount multiplied by the number of times that token has already shown up in the output so far. A token used four times loses four times as much as a token used once. A token never used loses nothing.
OpenAI's API reference describes the parameter as a number between -2.0 and 2.0, where positive values penalize new tokens based on their existing frequency in the text so far, decreasing the likelihood of repeating the same line verbatim. Negative values do the opposite and make repetition more likely.
A worked example, five steps
Take three candidate tokens and a penalty of 0.5. The raw scores are fixed to keep the arithmetic readable. Real models have tens of thousands of candidates, but the mechanism is the same.
Step | Raw score: "the" | Times "the" used | Adjusted score: "the" | Raw score: "and" | Winner |
|---|---|---|---|---|---|
1 | 3.0 | 0 | 3.0 | 2.5 | the |
2 | 3.0 | 1 | 2.5 | 2.5 | tie |
3 | 3.0 | 2 | 2.0 | 2.5 | and |
4 | 3.0 | 3 | 1.5 | 2.5 | and |
5 | 3.0 | 4 | 1.0 | 2.5 | and |
Nothing about "the" changed in the model. Its raw score stayed at 3.0 the whole time. The penalty alone pushed it below the alternative, and the longer the answer ran, the harder the push became.
That compounding is the point, and also the risk. A long answer that legitimately needs the same word many times, such as a product name, a variable name, or a technical term, is punished for being consistent.
Frequency penalty vs presence penalty
These two settings are usually offered together and are easy to confuse.
Frequency penalty scales with the count. Use it when the problem is a word or phrase used again and again.
Presence penalty is flat. A token is penalized once if it has appeared at all, no matter how many times. Per the same API reference, positive values increase the model's likelihood to talk about new topics.
In short, frequency discourages saying the same thing repeatedly. Presence encourages moving to new ground.
Which value to try first
There is no universal right number, and the effect varies by model. These are starting points to test against your own prompts, not rules.
Problem you see | Try | Why |
|---|---|---|
The model loops on one phrase | Frequency penalty 0.3 to 0.6 | Count-based pressure breaks loops early |
Answers circle one idea | Presence penalty 0.3 to 0.6 | A flat nudge toward new topics |
Output must repeat exact terms (code, names, legal wording) | 0 for both | Penalties fight consistency |
Creative writing feels samey | Small frequency penalty plus a higher top-p | Both widen word choice, so keep each gentle |
Change one setting at a time and compare five or ten outputs side by side. A single sample tells you almost nothing, because sampling is random.
When a frequency penalty is the wrong tool
If the model repeats itself because your prompt is vague, penalties only disguise the cause. The model is often repeating because it has nothing new to say, and a clearer instruction will fix that better than a number. The post on why AI models repeat themselves walks through the causes that no setting can touch.
Penalties also do not stop a model from ending cleanly. If you need output to end at a certain point, a stop sequence is the right control.
For the wider picture of the knobs that shape model output, see how AI models work.
Frequently asked questions
What is a good frequency penalty value?
Start at 0, and only raise it, usually to between 0.3 and 0.6, if you see repetition. Values near the upper limit of 2.0 tend to produce odd, avoidant wording because the model is being pushed away from ordinary words.
Does a frequency penalty make AI more creative?
Not directly. It reduces repetition, which can read as more varied. Real creativity depends mostly on the prompt and on sampling settings like temperature and top-p.
Is frequency penalty the same as repetition penalty?
They are close cousins. Some open-source tools expose a repetition penalty that works multiplicatively, while the API setting described here subtracts from the score. The goal is the same, but the numbers are not interchangeable between systems.
Can a negative frequency penalty be useful?
Rarely. A negative value encourages repetition, which can help in niche cases such as generating deliberately repetitive test data.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
