What Is Mixture of Experts? A Plain Guide
A mixture of experts model runs only a few of its sub-networks per token. That is why parameter counts stopped being comparable between models.
A mixture of experts model splits its network into many specialised sub-networks and runs only a few of them for each token. Instead of pushing every word through all the parameters, a small router picks which experts handle this particular token, and the rest sit idle. The result is a model with an enormous total parameter count that costs far less to run than that count suggests.
This is why parameter counts stopped being comparable between models, and why a "2.4 trillion parameter" headline tells you almost nothing about speed or price on its own.
The two numbers you need
Every mixture of experts model has two parameter counts, and confusing them is the most common mistake in reading model announcements.
Total parameters: everything stored in the model. Determines how much memory you need to hold it.
Active parameters: how much actually runs per token. Determines latency and compute cost.
Qwen3.8-Max, announced on 3 August 2026, is a useful worked example: roughly 2.4 trillion total parameters with about 95 billion active per token. So it stores like a 2.4T model and computes like a 95B one. Those are wildly different machines, and only the second number predicts what a request will cost you.
A dense model, the older design, has one number because everything runs every time. When you compare a 70B dense model to a 2.4T mixture of experts model, you are not comparing like with like in either direction. The dense model is smaller in storage and larger in compute per token than those headline figures imply.
How the routing works
The mechanism is less exotic than the name suggests.
Inside each layer, instead of one large feedforward block, there are many smaller ones. These are the experts. Before the token reaches them, a small network called the router scores each expert for this specific token and picks the top few, commonly two out of dozens or hundreds. Only those run. Their outputs are combined, weighted by the router's confidence, and the token moves to the next layer.
Two things about this are worth internalising:
The routing is per token, not per request. The word "Paris" in one sentence and the word "authenticate" in the next may be handled by entirely different experts, in the same forward pass.
The routing is learned, not designed. Nobody assigns one expert to French and another to Python. The specialisations emerge during training, and when researchers inspect them they are often not human-interpretable categories at all.
The word "expert" is doing a lot of misleading work here. These are not specialists in the way a cardiologist is. They are subdivisions that training found useful, and the divisions frequently correspond to nothing you could name.
Why anyone builds models this way
The economics. Model quality scales with parameters, and compute cost scales with parameters that actually run. Mixture of experts breaks the link between the two, letting you grow the first without paying proportionally for the second.
Concretely, if you want a model with substantially more capacity, a dense design means every request gets proportionally slower and more expensive. A mixture of experts design means you add experts, memory requirements rise, and per-token cost stays roughly flat because you still activate only a few.
There is no free lunch attached. The costs land in different places:
Cost | Where it shows up |
|---|---|
Memory | All experts must be loaded even though few run. This is why huge MoE models are impractical to self-host. |
Serving complexity | Efficient batching is harder when different tokens in a batch need different experts. |
Training stability | Routers can collapse toward a few favourite experts, wasting the rest. Mitigating this needs extra machinery. |
Uneven quality | Capability can vary in ways that are harder to predict than in a dense model. |
The memory row is the one with practical consequences for readers here. A 2.4T model being open weights does not mean you can run it. The weights still have to fit somewhere, and that somewhere is a cluster. If you are thinking about self-hosting, what it takes to run an AI coding model locally works through the arithmetic, and the answer for frontier MoE models is that it is not a workstation question.
What this changes for you in practice
Three things, if you are building on these models rather than training them.
Read active parameters, not total. When a release announces a very large number, find the active count before forming an opinion about cost or speed. It is usually in the technical detail rather than the headline. Qwen3.8's announcement is a recent example where the two figures differ by more than an order of magnitude.
Do not infer capability from either number. Both are architecture facts. Whether a model is good at your task is measured by trying it on your task, which is also the honest answer to what an AI benchmark tells you.
Expect the open-weights tier to be small models. The pattern in recent releases is a very large MoE flagship alongside a much smaller dense checkpoint. The flagship gets the announcement and the small one gets used, because it is the one people can actually run. This tension runs through the open-weight versus closed model tradeoff.
For the broader picture of how these systems work underneath, how AI models work covers the foundations, and what a foundation model is sets the terminology this sits inside.
FAQ
What does mixture of experts mean in AI?
An architecture where the network is divided into many sub-networks, and a router activates only a few of them per token. The model has a large total parameter count but runs only a fraction of it on any given input.
Is a mixture of experts model better than a dense model?
Neither is better in general. MoE buys more capacity at a given compute cost, and pays for it in memory, serving complexity and training difficulty. Which wins depends on whether you are constrained by compute or by memory.
Why do MoE models have two parameter counts?
Because storage and compute have been decoupled. Total parameters tell you how much memory the model occupies; active parameters tell you how much of it runs per token, which is what determines speed and cost.
Can I run a mixture of experts model on my own hardware?
Only small ones. Every expert has to be resident in memory even though few execute, so a trillion-parameter MoE model needs data-centre hardware regardless of how few parameters are active per token.
Are the experts specialised in topics?
Not in any way you could label. The specialisations are learned during training and generally do not correspond to human categories like language or subject matter. The name is a historical convention rather than a description.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
