AlexNet network wins image recognition contest by a wide margin

FLORENCE, OCTOBER 2012 - Around the European computer vision conference ECCV, the results of this year's ImageNet contest have been announced. The winner is a team from Toronto competing under the name SuperVision: Alex Krizhevsky, Ilya Sutskever and their PhD supervisor Geoffrey Hinton. Their program, AlexNet, has an error rate of 15.3 per cent. The runner-up is at 26.2 per cent, more than ten percentage points behind. Such a large gap is unusual in contests of this kind.

What did the program have to do?

The contest is based on ImageNet, the collection of sorted photos presented in 2009. Participants get more than 1.2 million photos to learn from, divided into 1,000 categories. After that, they get new photos they have never seen before. For each photo, the program may make five guesses about what it shows. If the right answer is not among them, it counts as an error.

How does AlexNet work?

AlexNet is a neural network: layers of small computing units with connections of different strengths, which learn by tracing their errors back through the network and adjusting the connections. That learning method, back-propagation, was described in 1986. What makes AlexNet special is the way it is built.

The first five layers work like filters that slide over the photo looking for a pattern, such as an edge or a change in colour. Each layer builds on the one before: edges turn into shapes, and shapes into parts of an animal or object. Three layers behind them combine everything into an answer. In total the network has eight layers, about 650,000 neurons and some 60 million connection strengths. All those strengths have to be adjusted. That takes an enormous amount of computing.

That is why the team trained the network on two NVIDIA graphics cards, each with 3 gigabytes of memory. The network did not fit on one card, so they split it across two. Training took five to six days. Thanks to their many small computing units, such chips do this kind of work much faster than an ordinary processor.

Two tricks also helped. The first is a simpler rule for what each unit passes on, which makes the network learn faster. The second is called dropout: during learning, the network keeps switching off a random half of its units, so that it does not come to rely on just a few of them and also works well on new photos.

The competition works differently. The runner-up combines several programs, each using features designed by researchers to describe a photo, such as colour patterns and the texture of the image. AlexNet works out its features by itself.

Looking back from 2026

AlexNet is seen as the first widely recognised use of deep networks with filter layers for image recognition. Yann LeCun, himself a pioneer in this field, later called it a turning point in the history of computer vision. The paper has been cited more than 198,000 times. The contest ended after 2017 because the task was considered solved.

At this one moment, three things came together that each already existed on their own: the learning method from 1986, the large photo collection from 2009, and the computing power of graphics chips, which had been programmable with CUDA since 2007. From here on, neural networks overtook the rest of AI. Three years later, Sutskever was one of the founders of OpenAI.