Chinese AI model Kimi K3.1 Pro ranks fourth in independent leaderboard

The Chinese AI company Moonshot AI released Kimi K3.1 Pro on 23 October. The testing firm Artificial Analysis tests AI models itself, always with the same set of ten tests covering programming, reasoning, science and independent work on a computer. The new model goes straight to fourth place in that leaderboard.

Kimi K3.1 Pro scores 55 points. Only three models score higher: Claude Opus 5.5 (58) and Claude Sonnet 5.5 (56) from Anthropic, and Muse Wonder Full (56) from Meta. GPT-6 Astra from OpenAI and Gemini 4 Argon from Google both come in at 53 points.

The same model, much more training

K3.1 Pro is not a new, bigger model. It is Kimi K3, which Moonshot released in July, with a lot of extra training on top. Moonshot had the model practise tasks in hundreds of fields, rewarding it when it carried out a task well. The size stayed the same: 2.8 trillion parameters, the numbers in which an AI model stores what it has learned. For each piece of text it uses 104 billion of them.

Even so, the score rose from 44 to 55 points. According to Artificial Analysis, no other model currently gets this much out of its size.

Biggest gains in independent work

The biggest jump is in tasks that the model carries out independently and step by step, using programs and files on a computer. That is the work of so-called AI agents.

• In a test with real work assignments from all kinds of professions, K3.1 Pro now ranks third, behind Claude Opus 5.5 and Muse Wonder Full.
• On Terminal-Bench 4.0, an AI does programming work by typing commands, the way a developer does. There the score went from 13 to 67.2 per cent. Artificial Analysis has not measured any model that scores higher. In its own measurement, Moonshot came to 65.7 per cent, just below Claude Opus 5.5.
• In automating work in business software that runs online, the score went from 58 to 73 per cent.

On knowledge and reasoning tests the progress is smaller. On Humanity's Last Exam, a collection of extremely difficult questions from experts in many fields, the score rose from 47 to 52 per cent. The model does make up answers less often, though. When it does not answer a question correctly, it now gives a wrong answer in 44 per cent of cases instead of admitting that it does not know. For K3 that was 53 per cent.

The same cost per task

The new model writes a little more text per task: 14 per cent more on average. At the same time, Moonshot has cut its prices by a third. As a result, a task from the test set costs almost the same as with K3: 1.98 dollars compared with 2.00 dollars.

At Moonshot, developers pay 2 dollars per million tokens (pieces of text a few letters long) that they send to the model, and 10 dollars per million tokens that the model writes back. K3.1 Pro can process very long texts in one go, up to a million tokens, and also understands images.

Free to download from 4 November at the latest

For now, K3.1 Pro only works through Moonshot itself: in the Kimi apps, in the programming tool Kimi Code and through a connection for developers. Moonshot promises to release the so-called weights by 4 November at the latest. That is the complete trained model. Companies and researchers can then download it and run it on their own servers, as happened earlier with Kimi K3.

According to Artificial Analysis, K3.1 Pro will then be the strongest open model in the leaderboard. The gap between the best open model and the best closed model will shrink from 12 to 3 points.

Hacking skills deliberately reduced

Moonshot says that for the open version it deliberately limited what the model can do when it comes to breaking into software. During training, the company saw that the model was getting better and better at finding and exploiting security holes. So it removed training material about cyberattacks, while keeping ordinary programming tasks.

In the company's own test with real, known security holes, a version without this change solved 90.5 per cent of the tasks. The released model reached 81 per cent, which is still far more than K3 (33.3 per cent). According to Moonshot, the model remains behind the strongest closed models in this area. The company also sees it as help for defenders: with an open model, many more teams can have their own software checked for security holes.

Caveats

The Artificial Analysis leaderboard sums up ten tests in a single number. On individual tests, other measurements sometimes come to different results, as with Terminal-Bench 4.0. Some of the figures that Moonshot itself gives, such as on fewer errors in financial work and the hacking test, come from the company's own tests. As soon as the weights are available, independent researchers can check them for themselves.

What does this mean for the road to the singularity?

This is a step on the road to AI that is getting better and more independent. The most striking thing is how the progress came about: not with a bigger model, but with much more targeted training of the same model. That shows there is still a lot of room in existing models, especially for independent work on a computer.

If the weights are indeed released on 4 November, researchers and companies around the world will be able to use a model close to the world's best, on their own hardware. In this way, what is now in the hands of a handful of companies comes ever faster within reach of everyone who wants to build and do research with it.

What is the technological singularity? Read the explanation →