What Is a State Space Model? SSMs Explained

A state space model, or SSM, is a type of neural network architecture that processes sequences by carrying a compact running summary forward one step at a time, instead of comparing every token to every other token the way a transformer model does. That single design choice is the whole story: it makes SSMs cheaper to run on long sequences, at the cost of being less precise at pulling back a specific fact from far earlier in the input. Mamba is the best-known modern SSM, and it exists specifically to challenge attention on very long context.

What a state space model actually does

The term comes from control theory, where a state space model describes a system by a hidden internal state that updates as new inputs arrive, plus a function that reads outputs off that state. Apply the same idea to language and you get a sequence model that reads tokens one at a time, folds each one into a fixed-size state vector, and produces an output from that state at every step. Nothing further back in the sequence gets re-examined; the state is the model's only memory of everything that came before.

Early SSMs for deep learning, such as S4, were built to solve a specific problem: plain recurrent networks like LSTMs are also sequential, but they struggle to carry information over long distances and train slowly since each step depends on the last. SSMs use structured mathematics from control theory to keep long-range signal in the state without that training bottleneck.

State space model vs transformer: why the cost difference is the whole point

A transformer's self-attention layer compares every token in a sequence to every other token, so compute and memory cost grows roughly with the square of the sequence length. Double the input and attention cost roughly quadruples. An SSM updates a fixed-size state once per token, so cost grows roughly linearly: double the input, and processing cost roughly doubles too.

To make that concrete: imagine feeding a model a 2,000-word document versus a 20,000-word document, a 10x jump in length. For illustration only, not a benchmark, a transformer's attention work goes up on the order of 100x, while an SSM's work goes up on the order of 10x. That gap is why SSMs are attractive for very long sequences such as full books, long audio transcripts, or genomic data: attention's cost eventually dominates everything else, while the SSM's cost keeps climbing at the same rate as the input itself.

This also shows up at inference time. A transformer keeps a growing cache of past keys and values, so memory use grows with context length. An SSM's state stays a fixed size regardless, which is part of why Mamba-style models can be markedly cheaper to run on long inputs.

The tradeoff: SSMs are not simply a free upgrade

The honest complication is that compressing an entire sequence into a fixed-size state means something has to give. Attention keeps a full, uncompressed record of every past token to compare against, which makes it very good at exact recall: finding one needle in a haystack of text, or copying a specific string verbatim from earlier in the context. An SSM's state is finite, so precise long-range recall and copying are harder for it by construction, not just as a training artifact. A widely cited 2024 paper, "Repeat After Me: Transformers Are Better Than State Space Models at Copying," found that transformers reliably beat SSMs on exact copying tasks, and that the gap holds even when the SSM has far more parameters.

This is why pure SSMs rarely get pitched as an outright transformer replacement. Most production systems mix SSM layers with a smaller number of attention layers for precise recall; Jamba is a widely cited example. This tradeoff also matters when you evaluate AI coding tools that advertise huge context windows: the underlying architecture affects not just how much text a model can accept, but how reliably it can find one specific detail buried in the middle of it.

The Mamba architecture in AI: what actually changed

Mamba's contribution was making the state update selective: the model learns, at every step, how strongly the current token should influence the state and how quickly older information should fade. Earlier SSMs used fixed update rules that treated every token the same way, which limited how well they handled language, where some words matter enormously and others barely register. Selectivity closed much of that gap, which is why Mamba, and its successor Mamba-2, became the reference point for SSM-based language models today.

Where you will encounter SSMs

• Long-document and long-audio models, where linear-time scaling matters more than perfect recall of one sentence.

• Hybrid language models pairing SSM layers with a thinner layer of attention, aiming for transformer-level quality at lower cost.

• Edge and on-device settings, where a fixed-size state and small memory footprint matter more than in a data center.

• Genomics and time series, where sequences run very long and exact recall matters less than overall pattern extraction.

SSMs connect to a broader problem worth knowing about: long inputs degrading a model's effective attention even inside a transformer, sometimes called context rot. SSMs sidestep the cost side of that problem by never expanding a cache, though they carry their own version of the same tension between how much a model can hold and how precisely it can use it. For the wider picture, see our overview of how AI models work.

FAQ

Is Mamba a state space model or something else?

Mamba is a specific, modern implementation of a state space model. It adds a selective update mechanism on top of the earlier S4-style SSM design, which is what let SSMs finally compete with transformers on language tasks rather than just synthetic sequence benchmarks.

Will state space models replace transformers?

Not outright, based on current evidence. SSMs are weaker than attention at exact copying and precise long-range recall, so most serious systems combine SSM layers with some attention rather than dropping attention entirely. Treat SSMs as a strong option for a specific tradeoff, not a strict upgrade.

Why does linear-time processing matter for long sequences?

Because attention's roughly quadratic cost compounds as sequences grow, while an SSM's roughly linear cost does not. At short context lengths the difference is minor. At the length of a full book, a long codebase, or hours of audio, that compounding gap becomes the deciding factor in whether a model is practical to run at all.

Are SSMs slower or faster to train than transformers?

Modern SSMs like Mamba are designed to train efficiently on parallel hardware despite being sequential in concept, using specialized algorithms that avoid the slow step-by-step training that hurt older recurrent networks. Training efficiency between the two architectures is closer than the phrase "sequential state update" might suggest.

What is a simple way to picture the state in a state space model?

Picture a running summary that gets rewritten slightly with every new token, rather than a full transcript re-read from the start each time. The summary is fixed in size no matter how long the input gets, which is exactly why it is cheap, and exactly why it cannot hold everything with equal precision.