NederlandsMistral releases new speech models that understand all official EU languages
The French AI company Mistral has released Voxtral 3: AI models that turn speech into text and read text aloud, plus software to build a voice assistant with them. They work in more than fifty languages, including all 24 official EU languages, and can also run on a company's own servers.
Talking is the easiest way for people to ask for something. For computers it was long the hardest: understanding what someone says, who is speaking, and then answering in a natural voice. The French AI company Mistral is now taking a new step here with Voxtral 3, a family of speech models.
Four models and a toolkit
Voxtral 3 consists of four AI models. Two of them turn recorded speech into text: a large version and a smaller one, Voxtral Mini Transcribe 3, with 4 billion parameters, the adjustable numbers of an AI model. A third model does this live, while someone is talking, with less than half a second of delay according to Mistral. The fourth model, Voxtral Speech 3, does the reverse: it reads text aloud in a natural voice, and you can control emotion, pace and emphasis.
In addition, Mistral is releasing a software package for building a voice assistant with these models. Such an assistant listens, thinks, can look something up or consult a system itself, and then answers out loud. According to Mistral, each turn in such a conversation takes less than a second. The assistant works over the phone and in the browser or an app.
Who said what?
The models can do more than just write out words. They keep track of who is speaking, for example in a meeting with five people. You can also give them a list of difficult words in advance, such as names of people, products or technical terms. On recordings of conversations about company results, the large model wrote such words correctly in almost 95 per cent of cases according to Mistral, compared with just over 81 per cent without such a list.
In keeping track of who says what, the large model made mistakes in just over 18 per cent of the words, according to Mistral. That is fewer than the speech models of ElevenLabs (23 per cent) and Google (just over 38 per cent) in the same test. These are Mistral's own measurements; independent tests usually follow in the weeks after such a launch.
The speech recognition works in more than fifty languages, including all 24 official languages of the European Union. Reading aloud is possible in more than 35 languages.
Cheap and in Europe
Transcribing an hour of speech costs about 21 US cents with the smallest model, and about 48 US cents with the live model. Companies can use the models through Mistral's servers in Europe, or run them on their own computers. For hospitals, local councils and banks, for example, which do not want sensitive conversations to be processed elsewhere just like that, this makes it easier to use speech technology.
What does this mean for the road to the singularity?
Speech is the shortest route between a person and an AI. The better a model understands who says what, and the faster it answers out loud, the more AI becomes a conversation partner instead of a program you type text into. Such models give AI agents ears and a voice: soon they will be able to make phone calls, move an appointment or write up a meeting. The most interesting effect may lie in the combination of languages and knowledge. A voice assistant that understands fifty languages, backed by a language model that knows every field, can let a Polish farmer consult a Spanish soil expert in their own language, while the AI thinks along about what they both overlook. That Mistral offers this from Europe, also to run yourself, puts that future within reach of many more organisations.
What is the technological singularity? Read the explanation →
