PW Pete Warden On The Data Exchange with Ben Lorica

“I like to think of a GPU as like trying to use an oil tanker to go and do your shopping. They're fantastic for these large training jobs, but they're complete overkill for inference.”

The Data Exchange with Ben Lorica · AI Research & Frontier Labs · November 2026

“I like to think of a GPU as like trying to use an oil tanker to go and do your shopping. They're fantastic for these large training jobs, but they're complete overkill for inference.” — Pete Warden, The Data Exchange with Ben Lorica

Context

Ben Lorica asked about AI hardware and whether Nvidia's dominance will last. Warden says GPUs are built for large batches and training, while inference puts a different load on hardware.

Transcript

The Data Exchange with Ben Lorica Around 34:27 into the episode
Pete Warden

Though CPU could actually be pretty fast. Obviously,

Ben Lorica

the NVIDIA answer is why. We have all the HBM on high-bandwood memory right on the GPU, right?

Pete Warden

And I like to think of GPUs as, you know, they're designed for large batches and training. I like to think of a GPU as like trying to use an oil tanker to go and do your shopping. You know, it's, it's, they're fantastic for these large training jobs, but they're complete overkill for inference. Inference has a very different shape to its, you know, to the workload that it places on hardware.

Ben Lorica

One thing that I've kind of started to pay attention lately. And it's very, very early as a high-bandwidth flash.

Pete Warden

Absolutely. Right?

Ben Lorica

So, what's your take?

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Data Exchange with Ben Lorica