Why AI Safety Filters Miss Split-Up Tasks
A model that refuses a harmful request outright will often complete the same request delivered as six harmless-looking steps. Here is the mechanism and the defence that measurably works.
Ask a model to do something clearly harmful and it refuses. Break the same job into six steps that each look dull and reasonable, and it will often do all six. That is the short version of why AI safety filters miss split-up tasks, and it is not a bug in one product. It follows from where the filter sits.
Where AI safety filters miss split-up tasks
Most safety checks run on the request in front of them. A classifier reads the prompt, sometimes the prompt plus the draft response, and decides whether that exchange is acceptable. It is good at this. What it does not do is reason about where a conversation is going.
Consider the structure of a disinformation operation rather than its content. Step one: write a plausible biography for a regional reporter. Step two: summarise six local news stories in a neutral register. Step three: suggest a posting cadence that looks organic. Step four: translate all of it. None of those requests is refusable. A freelance journalist might send all four in a week. The harm is in the assembly, and the assembly never appears in any single message.
Researchers call this a decomposition attack. The framing in the literature is that a malicious goal is split into benign subtasks that slip past refusals, and that current defences are narrow because they detect harm in the immediate prompt without reasoning about long-range intent.
How big the gap is
This has been measured. A benchmark called DecompBench, presented at ICML and NeurIPS in 2026, tested agents on harmful tasks in two forms: as one monolithic request, and as decomposed variants. The finding was a large asymmetry. State-of-the-art agents refused the monolithic versions at high rates and refused the decomposed versions at significantly lower rates. Same goal, same agent, different packaging.
Anthropic reported the same shape of result from the outside. In independent testing with the Institute for Strategic Dialogue and the McCain Institute, covering the US, Israel and Brazil, it found that Claude reliably refused single harmful requests but could miss campaigns split into individually harmless tasks. That is a vendor publishing a gap in its own defences, which is worth more than a benchmark.
If you want the broader picture of how this testing happens before a model ships, we covered who tests an AI model before release and what attack success rate measures.
What actually helps
The defence that shows real numbers is not a better per-request classifier. It is a monitor that reads the whole conversation.
A 2026 paper from NYU and EPFL (arXiv 2506.10949) proposes exactly that: an external sequential monitor that evaluates the accumulated context rather than each message in isolation, and asks whether the trajectory adds up to something the individual steps do not. The authors report lightweight sequential monitors are highly effective against decomposition attacks and viable in deployment, with up to a 93 percent defence success rate. They released datasets and code.
The word lightweight matters. A monitor that costs as much as the model it guards does not ship. A small model reading a rolling summary of the session can.
If you are building on a model
The practical takeaway is that the provider's refusal behaviour is not your abuse policy. It is one layer, aimed at single requests, and the vendor has told you where it ends.
Keep state per user, not per request. If your app only ever sees one message at a time, you have inherited the exact blind spot described above.
Monitor at the session or account level. The signal is in the sequence: forty persona drafts from one account in an hour is a pattern, and each draft on its own is not.
Rate-limit the composable things. Persona generation, bulk translation and scheduling are the reusable parts of these operations, and limits on them cost legitimate users almost nothing.
Write the prohibited outcomes down, not just the prohibited words. A policy that lists banned topics misses an operation assembled from permitted ones.
A guardrail model sitting alongside your main model is the usual shape of this, and the useful change is giving it the conversation rather than the message. Our pillar on AI risks places this among the other failure modes worth planning for.
Frequently asked questions
Does this mean AI safety filters do not work?
No. They work well on what they are built for, which is a single request with harmful content in it. The failure is specific: harm that exists only in how several acceptable requests combine. Treating a filter as a complete abuse system is the mistake, not using one.
Is splitting a task up always an attack?
No, and that is the difficulty. Decomposition is also how ordinary work gets done, which is why it is hard to filter. The distinguishing signal is intent across the whole sequence, which is why monitoring the trajectory beats scoring the step.
Why does a model refuse harmless questions, then?
Over-refusal and under-refusal are separate problems that pull in opposite directions, and tightening one usually worsens the other. We went through the over-refusal side in why AI refuses safe questions.
Do I need this if my product only writes marketing copy?
If it generates text at volume on a user's behalf, yes, because that is the capability these operations want. The question is not what your product is for. It is what it can be assembled into.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
