Granite Switch 4.1: chat, decisions, speech in one model

Granite Switch 4.1 is a set of IBM Granite models that answer chat requests, make fast structured decisions and transcribe speech from a single endpoint. IBM published it on 15 October 2026, in two sizes: 8B and 30B. The pitch is that you stop running a chat model, a classifier and a speech model as three separate services.

Everything below comes from IBM's announcement on the Hugging Face blog, read on 17 October. Where the post gives no number, this article says so instead of guessing.

What Granite Switch 4.1 includes

Three behaviours share one set of weights:

• Chat. Ordinary natural-language responses from the Granite base model.

• Decisions. A general-purpose Decision adapter, plus a separate Guardian adapter aimed at safety-related decisions, such as deciding whether an input should be allowed through.

• Speech. A speech recognition component, Granite-Speech-5.0-Turbo-CTC, which IBM calls the fastest model on the OpenASR Leaderboard. No speed figure is given in the post.

The adapters are Activated LoRAs. If you have not met the idea, what LoRA is in AI covers the basics: small add-on weights that change a base model's behaviour without retraining all of it. The "activated" part matters here. IBM says the cache built while the base model generates a reply can be reused when an adapter is called afterwards, so a decision made right after a chat turn does not pay to re-read the whole conversation.

IBM also says the combined model uses about the same GPU memory as the base model alone.

The numbers IBM does give

The post is light on benchmarks, so the short list is short:

• A custom 3B Granite Switch model in a Doom demo made "60 - 100 decisions / second", against 10 decisions per second reported for a competing model, Jev.

• In a race on a five-step retrieval workflow, the 30B model finished faster than a 27B model from Jev. No timings are published.

• The Guardian adapter "on average" outperforms four named decision models on safety-related decisions. No scores are published.

Treat these as vendor claims you could reproduce, not results you can cite. A demo that makes 60 to 100 decisions per second says the plumbing is quick. It does not say the decisions are good.

How you would run it

According to the post, you can serve it in vLLM using a plug-in from the Granite Switch repository, or use native support in llama.cpp and downstream tools. The model can stand in for calls to a /v1/systemone decision API while still serving normal chat requests, sharing a prefix cache between the two. You can also compose your own Granite Switch variant by mixing adapters from IBM's Granite Libraries catalogue. A software kit for training your own adapters is described as planned, not released.

If serving details are new to you, what an inference engine is explains what vLLM and llama.cpp actually do.

What the post does not say

Before this goes anywhere near a product, three gaps are worth closing:

1. Licence. The announcement does not state one. Read the model card for each size before use, especially if you sell the result.

2. Scores. No accuracy numbers for the decision adapters or the speech component. If you care about transcription quality, you will have to measure word error rate on your own audio.

3. Cost against a split stack. One model with the memory footprint of the base is attractive, but IBM publishes no side-by-side with a small chat model plus a separate classifier.

Who should look at it

If you run an agent that spends most of its calls on routing, filtering and yes-or-no checks, a model built for fast decisions that shares a cache with the chat model is the interesting part. If you are a small language model shop already self-hosting, the 8B size is the one to try first. If you only call hosted APIs and never touch model files, there is little here to act on yet.

For where this sits among the week's other releases, how to keep up with AI news is the running guide.