“Fei-Fei Li, who ran the competition, when she heard a neural network had won the image recognition competition, and not just won it but crushed it, her words were: it was like learning that the land speed record had been broken by 100 miles an hour by somebody driving a Honda Civic.”
Search Engine · Tech Strategy & Big Tech · October 2026
Witt is explaining how dead neural networks looked as a field before AlexNet, a neural net that ran on an NVIDIA GPU, won the 2012 image recognition competition. Within a couple of years nearly every entry was a neural network.
Transcript
…He said, God, if quantum physicists are telling me that they're constrained by computing and I can provide it to them cheaply, it just has to be good. There has to be some way that I'm going to unlock some branch of science somewhere that will ultimately be extremely profitable for my company.
…its own. Neural networks would ultimately be the right answer. But the problem was they've been tried and had failed so many times that by the 2000s, the consensus was that they would never work. The handful of researchers still betting on them were given a little more respect than flat earthers.
Within AI, neural networks were seen as like a dead technology. So those guys were also mavericks. They were really mad scientists. People did not think biologically inspired software was going to work. They thought it was a joke. I have the 1100-page third edition of the AI standard AI textbook from 2011. It's 1,100 pages long. It looks like the phone book. There's 16 pages in there about neural networks. Oh, wow. Like you could get a degree in AI and not ever touch a neural network. You could be a PhD and never use a neural network. In fact, most PhDs would only have limited experience with them. And PhD advisors would have discouraged you strongly from pursuing any research in this technology, which was seen as antiquated and which was seen as dead. But an ACE programmer named Alex Krushevsky coded a neural net called AlexNet that ran on the GPU. And that was the one that all of the AI scientists saw, and there was a mass conversion immediately. So the year that he entered his neural network into the image recognition competition, it was the only neural network…
And how quickly does NVIDIA become aware of this? How quickly does it change his company's fortune? Like when does the wind reach his sale?
The wind reaches his sales when Hampton's group, Jeffrey Hinton and Alex Prudievsky, get aqua hired by Google. And they come in and Google at that time has the largest platform of computers, probably outside of a government platform in the world. This giant distributed cluster of CPUs. And those…
Welcome to GTC 2014. This is what started it all. CUDA and the invention of GPU computing launched this event. As I was showing up this morning, a good friend said, This is the Woodstock for computational mathematicians. I hope it turns out the same way. I hope it turns out the same way.
More from Search Engine
