Dwarkesh Patel
Hosts the Dwarkesh Podcast, a long-form interview show on AI progress, timelines, and the people building frontier models, along with economists and historians.
“If you had more lack of structure, having a bunch of tiny TPUs makes a lot of sense. Whereas if you just have huge matrix multiplications, you'd think: why don't we just avoid the cost of individual SMs with their own registers and work schedulers, and make one huge thing that amortizes those costs across the whole thing?”
Dwarkesh Podcast · AI Research & Frontier Labs · May 2026
Context
Restating the GPU-versus-TPU architectural tradeoff in his own words, after Pope walked through how systolic arrays scale. The framing makes it a question about the workload rather than the hardware: irregular work rewards many small flexible units, uniform work rewards one big one.
Transcript
So you're suggesting the tensor core within a streaming SM is analogous to an MXU. Yeah,
it's very, very similar. Yeah.
I see. And so if you had more lack of structure, having a bunch of tiny TPUs makes a lot of sense. Whereas if you kind of just have huge matrix multiplications, you're like, why don't we just avoid the cost of having the individual SMs with their own registers and work schedulers and things like that? Why don't we just make a huge thing and amortize those costs across the whole thing?
And I mean, I think this shows up in how large you can grow things. We've sort of seen this theme, especially with a systolic array, where larger systolic array amortizes the register file costs better. This sort of design allows you to have larger systolic arrays, whereas the sort of GPU design…
Right, but also you might have to move across less area.
Which I mean is also saving its energy sensor. So data ends up moving like if you can operate entirely within an SM, the data movement is much smaller. But then the moment you want to operate across SMs, it becomes sort of more complicated and expensive. So
More from Dwarkesh Podcast
