NederlandsLarge open AI model from DeepSeek now fits on a computer with 128 gigabytes of memory
DeepSeek's new open AI model is too big for an ordinary computer. The company Unsloth made a heavily reduced version that runs on a powerful laptop or mini computer with 128 gigabytes of memory. This costs some accuracy.
The Chinese AI company DeepSeek released a new version of its fast model this month: DeepSeek V4.1 Flash 1017. According to DeepSeek, it is a big improvement at programming and at tasks in which the model takes steps on its own. On a test in which an AI model has to carry out tasks in a computer terminal by itself, the score rose from 31 to 42 points, according to the company.
The model is open: anyone may download and use it for free, under the permissive MIT licence. The catch is that the model is huge. It has more than 550 billion parameters (the numbers in which an AI model stores its knowledge), plus a separate memory part of almost 200 billion. The full download is more than 500 gigabytes. Something like that normally only runs in a data centre.
Made smaller, just like a photo
The company Unsloth, which makes open AI models smaller and faster, released reduced versions on 19 October. The smallest is 233 gigabytes, less than half the original. This is possible with a technique called quantisation.
A clever trick helps too. The model's separate memory part, a kind of reference book of common word sequences, does not have to be in the working memory. It can stay on the fast storage drive (SSD). As a result, according to Unsloth, the smallest version needs about 128 gigabytes of working memory. The most powerful Apple laptops have that, as do mini computers for AI developers, such as Nvidia's DGX Spark. But a hefty gaming PC can also do the job, as long as it is fitted with 128 gigabytes of RAM; for the larger versions that memory is not enough. A separate graphics card is not required: according to the guide, the model also runs on the processor alone, and a graphics card is mainly about speed. And speed is the downside: on home hardware, answers come in considerably more slowly than you are used to from ChatGPT, which runs in data centres full of specialised chips.
What do you give up?
Rounding off costs accuracy. Unsloth measures this by having the reduced model and the original predict the same texts. With a slightly larger version of 250 gigabytes, the reduced model picks the same next word as the original in about 87 to 88 out of 100 cases. With the version of just over 500 gigabytes, it is 98 out of 100.
That says something, but not everything. It is a comparison on a few test texts, not a measurement of how well the model carries out real tasks. Users and independent testers are now finding out how the smallest versions perform at programming and other tasks.
What does this mean for the road to the singularity?
A few years ago, the best AI models could only be reached through the servers of a few large companies. Now someone with one powerful computer at home can run a model that is among the strongest open models in the world. This means researchers, small businesses and hospitals can use such a model without sending their data outside.
Every step in making models smaller makes smart AI cheaper and more widely available. And the more people can experiment with it, the faster new applications and improvements arrive. If shrinking continues at the same pace, a model of this level may simply run on an ordinary laptop in a few years.
What is the technological singularity? Read the explanation →
