Unsloth shrinks a large open AI model from Alibaba and, by its own tests, makes it more accurate than other versions

Large AI language models are huge. Qwen3.8-Flash-Next, an open model from the Qwen AI team at the Chinese company Alibaba, has 125 billion parameters: the numbers in which the model stores its knowledge. In its full version the file is about 354 gigabytes. That does not fit on any ordinary computer. Anyone who wants to use such a model at home or in the office, without sending data to a data centre, therefore needs a smaller version.

The company Unsloth makes such smaller versions. On 4 November it released a new series for Qwen3.8-Flash-Next, made with an improved method that it calls Dynamic 3.1. Co-founder Daniel Han announced the new files on Hugging Face, a website where AI models are shared for free.

How to make a model smaller

The technique is called quantisation. Each number in the model is stored with fewer digits after the decimal point, roughly like rounding 3.14159 to 3.1. The file becomes much smaller, but the model makes slightly more mistakes. The trick is to round where it does little harm and to stay precise where it matters.

Unsloth's smallest version is just over 66 gigabytes, about a fifth of the original. It still needs a powerful computer with a lot of working memory. The files work with free programs such as llama.cpp and LM Studio.

What is new is that one part of the model, a kind of lookup table, can be chosen separately in four sizes: from 28.8 to 102.4 gigabytes. The smaller that table, the larger the difference from the original. Users can therefore choose for themselves between a smaller file and a more accurate model.

What the tests show

According to Unsloth, the new versions are the most accurate for their size on almost all tests, compared with other shrunken versions of the same model. For files of the same size, they score 15 per cent higher on average. Unsloth mainly tested tasks in which the model works independently, such as fixing programming errors and carrying out commands on the computer, as well as questions on mathematics and science. For these tests, the models together wrote tens of billions of tokens, the word fragments that AI models work with.

All figures come from Unsloth's own tests. The company itself also describes how sensitive such measurements are: after it improved the software that checks the answers, thousands of answers that had first been marked wrong were marked right. Independent measurements will have to show how large the gain is.

What does this mean for the road to the singularity?

An AI model only becomes truly useful when people can run it. Every improvement in shrinking models means that a powerful model fits on less expensive equipment. That gives researchers, small companies and enthusiasts the chance to work with such models themselves, to build with them and to check them, without depending on a large data centre. In this way, strong AI comes within reach of more and more people.

What is the technological singularity? Read the explanation →