What Is a Decision Model in AI? Explained

A decision model takes some input and a set of answers you define in advance, and returns a probability for each answer. It does not write a sentence. Your code reads the numbers and branches. Because nothing is generated, the answer can never be an option you did not define, and there is no JSON to repair.

That is the whole idea. It sounds small, and it removes the single most annoying failure mode in production AI work: the model that returns something almost valid.

How it differs from asking a chat model

Ask a large language model "is this support ticket about billing, yes or no" and it writes the word yes, one token at a time, after deciding how to phrase itself. You then parse that. Usually it says yes. Sometimes it says "Yes, this appears to be billing-related." Occasionally it says yes in a code fence. You write a parser, the parser grows cases, and the cases are the bug.

A decision model skips the writing step. One pass through the network, probabilities out.

The practical consequence is calibration. A probability of 0.91 is a number you can threshold. "Yes" is not. You can ask a chat model to return a confidence score, and it will produce one, but it is producing a token that looks like a number rather than reading one off a distribution.

The three question shapes

The format that has settled across these models covers three kinds of question:

• Binary: true or false, returned as a probability between 0 and 1.

• Choice: pick one of between 2 and 255 options you supply, with a confidence per option.

• Score: place the input on an ordered scale of between 2 and 10 levels, for things like severity or sentiment.

Almost every routing, triage, moderation and guardrail problem is one of those three wearing a costume.

What shipped, and what it costs

Four of these landed inside a fortnight, which is why the category suddenly has a name. We covered two of the launches as news at the time, Cloudflare's Clef release and the Decisions API preview; this is the category rather than any one product.

Clef is post-trained on Qwen3.8-27B and Clef-flash on Qwen3.5-9B. Both add a vision encoder and a 64k context window, and both use the same request and response format as Jev, which is the part that makes the category portable rather than four incompatible products.

On latency, one independent write-up measuring from Italy over the public REST API reported a median of 191 to 205 ms for Clef-flash and 524 to 726 ms for Clef. Vendor-quoted figures circulating elsewhere are considerably lower, which is normal, since those are measured next to the hardware. Use the vendor number for capacity planning and the independent number for what your users will feel.

When a decision model is the right tool

The honest test has two sides, and a decision model has to beat both.

Against an if-statement. If the rule can be written down and will still be correct next month, write the rule. A regular expression that checks whether a string contains an order number does not need a model. Decision models earn their place when the input is messy natural language, an image, or something where the rule would have forty exceptions.

Against a chat model. If the question is open-ended, or the output is the point, you want the chat model. Summarising a ticket is generation. Deciding which queue the ticket goes in is a decision.

The layer where they win is the boring middle: high-volume, low-stakes-per-call classification that currently runs through a general model because that was the only option. Routing. Tagging. First-pass moderation. Deciding whether an agent's next step needs a human. That last one is the case worth watching, because a guardrail that costs a fraction of a cent and answers in 200 ms is a guardrail you can afford to put on every step rather than every tenth.

The catch nobody mentions

You still have to define the options, and defining the options is the hard part. A decision model cannot tell you that your six support categories overlap, or that 30% of real tickets belong in a seventh you never created. It will confidently distribute probability across the six you gave it.

The failure mode moves from invalid output to a badly specified question, which is quieter and worse. Before wiring one up, run a few hundred real inputs through it and look at the cases where the top probability sits near the second, because that is where your taxonomy is wrong. The same discipline described in prompting a model to avoid making assumptions applies, except here the assumption is yours.

FAQ

Is a decision model just a classifier?

Functionally yes, and that is a feature. The difference from a classifier you train yourself is that you do not train it. You describe the options at request time, which means you can change the taxonomy without a retraining cycle.

Can I run one on my own hardware?

Several, yes. Clef, Clef-flash, pplx-decider-v1-27b and Strands Decider 2B all publish weights. Strands Decider 2B is small enough to run on a consumer GPU. Whether self-hosting is cheaper than an API at your volume is the same calculation as self-hosting any open-weight model.

Do they replace prompt engineering?

They shrink it. There is no output format to coax, so the work moves entirely to describing the input and defining the options well. The general principles in the prompt engineering guide still apply to how you frame the question.

Why no output charge on some of them?

Because there is essentially no output. The model emits a probability distribution rather than a token stream, so the usual output-token meter has nothing to count.

Sources: Cloudflare's Clef announcement coverage and an independent deep dive with measured latencies and pricing, both 1 October 2026.