NederlandsMixture of experts
A mixture of experts is an AI model built like a team of specialists. For each piece of text it picks only the experts that are needed, which saves computation.
A mixture of experts, MoE for short, is an AI model built like a team of specialists. Instead of the whole model computing for every piece of text, it picks a few experts from a large group. Only those do the computing. It is comparable to a hospital: for a broken wrist you do not visit every specialist either, but go to the right one.
How does it work?
A mixture-of-experts model contains many small sub-networks, the experts, and a component that makes the choice, the router. For each piece of text, the router decides which experts are used. The rest stay idle for that round.
This allows such a model to be very large, with a lot of knowledge, while doing only part of the computation per word. That makes it faster and more economical than a model of the same size in which everything always takes part. The model does have to keep all those experts in memory, and that takes a lot of memory space.
Why it matters
Many new language models are built this way. That has consequences for the chips they run on. Such a model contains roughly two kinds of steps, and the step in which the experts do their work mainly requires the chip to reach its memory very quickly. The chip company d-Matrix used that and split the work across two kinds of chips, with ordinary graphics chips and its own chips each doing what they are good at (link). That way answers appear faster.
The advantage does not hold for every model: the approach only works for models that use experts, although there are more and more of them.
Articles about Mixture of experts
-
Anthropic builds FlashProve: cryptographic proof of which AI model produced an answer
Anthropic has released FlashProve, an open-source research prototype that uses zero-knowledge cryptography to show which model generated an answer, without revealing the model weights.
-
Free coding model for the laptop was largely developed by AI itself
The AI company Recursive has released Flint, a free AI model that helps with programming and runs on an ordinary laptop. According to the company, its own research AI did a large part of the work: a first step towards AI that improves itself.
-
d-Matrix splits the work of an AI model across two kinds of chips for faster answers
At a major data centre conference, the American chip company d-Matrix showed how an AI model answers faster when ordinary graphics chips and its own chips each do the work they are good at. The company now also supplies complete computing clusters.
-
Meta releases Muse Spark Open: a large AI model for programming that also runs on your own computers
Meta has released Muse Spark Open, a large AI model for programming and independent agent tasks that anyone can download. According to Meta, it is the strongest American open model so far, and it even runs on two Mac Studios.
-
Gemini 4 Argon widely available: Google back among the leaders
Google has rolled out Gemini 4 Argon widely. On paper it beats GPT-6 Astra and Claude Opus 5.5 on 13 of 19 benchmarks, but independent measurements put it alongside the competition, not above it. The biggest leap is in price, long context and science.
