NVIDIA lets graphics chips do ordinary calculations too

THEALE, NOVEMBER 2006 - The chipmaker NVIDIA has announced CUDA, a new way to use a computer's graphics chip for all kinds of calculations. Until now, these chips have mainly been used to create images for games and design software. With CUDA, programmers can also give them ordinary computing work. According to NVIDIA, this means difficult problems can be solved up to 100 times faster than with the usual approach.

One fast calculator or lots of small ones

An ordinary processor in a computer, the CPU, is like a skilled all-rounder. It carries out a few tasks at the same time and does them at lightning speed, one after another. A graphics chip, the GPU, is built differently. To put an image on the screen, the colour of each of the millions of tiny dots has to be calculated. These are all simple sums that look alike. That is why a graphics chip consists of a great many small computing units that work at the same time. The new GeForce 8800 has 128 of them, each running at 1.35 gigahertz.

The missing piece: an ordinary language

Many problems outside graphics also consist of huge numbers of similar sums on lots of data. Anyone who wanted to use a graphics chip for that had to disguise the work as an image problem. That required knowledge of graphics programming. CUDA removes that barrier. NVIDIA supplies a compiler for the well-known programming language C. A compiler translates a programmer's text into instructions the chip understands. So a programmer simply writes C, and the chip does the computing.

Thousands of threads at once

The work is divided into threads: small sequences of instructions that each carry out a piece of the calculation. Thousands of threads run at the same time. In image processing, for example, the threads do not need to talk to each other, because each dot is processed separately. For harder problems there is a shared memory on the chip, the Parallel Data Cache. This lets the 128 computing cores work together. The CUDA development kit is available to developers and researchers who sign up.

Companies such as Acceleware and Headwave expect it to benefit their customers.

Looking back from 2026

CUDA became publicly available in February 2007. In 2012, the Toronto team behind AlexNet trained its network on two NVIDIA graphics cards and won an image recognition contest with it. Training a neural network consists of enormous numbers of similar multiplications, exactly the kind of work a graphics chip is built for. From around 2015, the development of CUDA focused more and more on machine learning and neural networks.

In this way, a technology for images and games, together with a programming language anyone could use, ended up powering the training of the largest AI models.