What Is Attack Success Rate in AI Security?
A vendor reporting a 0.7 percent attack success rate has told you something real and something incomplete. Knowing which is which is the whole skill.
Attack success rate, usually shortened to ASR, is the share of attack attempts that get past a system's defences. If you fire 1,000 crafted prompts at a model and 7 of them make it do the forbidden thing, the attack success rate is 0.7 percent. Lower is better, and the direction is the only part that is simple.
The number appears in model safety reports, security benchmarks and vendor comparisons, and it is quoted far more often than it is interrogated. Two systems reporting the same percentage can differ by orders of magnitude in how much protection they actually give you.
The denominator is the number that matters
An attack success rate is a fraction, and almost all the information lives underneath the line.
Google's October 2026 safety and security report for Gemini 4 Argon states that on Gray Swan's Indirect Prompt Injection benchmark, attack success across 15 adaptive attempts fell to 0.7 percent. Read that phrase slowly, because every word is doing work.
Indirect means the attack arrives inside content the model is processing, not typed by the user. A poisoned web page, a document, an email.
Adaptive means the attacker saw what failed and changed approach. This is much harder to defend than a fixed list of attacks.
15 attempts is the budget the attacker was given. Not one, not unlimited.
So the claim is: against an attacker who gets 15 informed tries, roughly 7 in 1,000 attack scenarios still succeed. That is a strong result, measured in a demanding way. It is also a statement about a specific adversary budget, and real attackers are not capped at 15 tries.
Four things the percentage does not tell you
Hidden variable | Why it changes the meaning |
|---|---|
Attempt budget | 1 try and 100 tries produce very different rates from the same system |
Who wrote the attacks | A fixed public set is easier than a red team adapting against you |
What counts as success | Leaking a secret and producing a rude sentence are both 'success' in some benchmarks |
What was connected | A model with no tools cannot take a damaging action regardless of ASR |
That last row is the one that bites builders. The benchmark measures the model. Your risk is the model plus every tool you handed it, which is why a near-perfect ASR still leaves you exposed if a successful instruction reaches something that can spend money or delete records. This is the confused deputy problem: the model is tricked into misusing authority it legitimately holds, and no resistance score removes the authority.
How to read a reported figure in practice
Find the denominator. If the attempt count is not stated, the percentage is decoration.
Check whether attacks were adaptive. Static sets age badly, because defences get tuned against published attacks.
Check the success definition. Ask what the benchmark counted as a win.
Compare only within one benchmark. Two vendors quoting ASR from different suites are not comparable, and neither is the same vendor across versions of a suite.
A figure that survives those four questions is genuinely informative. Most marketing figures do not get past the first.
Measuring your own
Your application's rate is the one that governs your risk, and you can measure it in an afternoon. Write 30 to 50 attack cases that match how content actually reaches your system: a document with an instruction buried in a footnote, a web page with white-on-white text, a form field with a fake system message. Define success in terms of your system, for example the assistant calls the delete tool or reveals the system prompt. Run each case five times, because model output varies.
Then do the part that most people skip: rerun the whole set after every prompt change. A defence that works today and is never re-tested is an assumption. The exercise is a narrow form of red teaming, and the narrowness is the point, because it gives you a number you can watch move.
If injected instructions are a new idea, the mechanism is covered in the plain-English version of prompt injection, and the image-based variant shows why screening text alone is not enough.
Questions people ask
Is a 0 percent attack success rate possible?
Against a fixed attack set, yes, and it means the set is exhausted rather than the system being invulnerable. Against an adaptive attacker with an unbounded budget, no. Treat a reported zero as a prompt to ask about the denominator.
Is a lower ASR always better?
Lower is better at equal strictness, and strictness is adjustable. A system that refuses far more legitimate requests can post a lovely ASR while being worse to use. The honest pairing is attack success rate alongside a false-refusal rate, and reports that give only one are giving you half the picture.
How does this relate to how a model decides anything?
An attack succeeds when injected text outcompetes your instructions for influence over the next token. There is no privileged channel that marks your words as the real ones, which is why the defence has to sit outside the model. How AI models work covers why that is structural rather than a bug waiting to be patched.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
