DA Diogo Almeida On a16z Podcast

“Are you really telling me that math is solved, or even two years ago GPQA, Google-proof question answering, is solved, but we still can't handle a drive-through? It's a very hard thing to hold in your head at once.”

a16z Podcast · Startups & Venture · September 2026

“Are you really telling me that math is solved, or even two years ago GPQA, Google-proof question answering, is solved, but we still can't handle a drive-through? It's a very hard thing to hold in your head at once.” — Diogo Almeida, a16z Podcast

Context

Almeida argues that the real test of AI is whether it can automate ordinary productive work, not whether it can top hard benchmarks. Casado pushes back right after, suggesting the real world is long-tailed and the training data for it simply isn't there yet.

Transcript

a16z Podcast Around 22:23 into the episode
Martin Casado

And so you think that the measure that we should have is to what extent can you automate actual productive tasks? That the overpromise and under deliver, that's

Speaker 4

the dimension in particular you're talking to. The ability to automate tasks.

Diogo Almeida

I think in my heart, it's like cool sci-fi. And I think that that is the canary in the coal mine for cool sci-fi. Like, are you really telling me that math is solved or like even two years ago, GPQA, that Google proof question answering is solved, but we still can't handle a drive-through, right? Like it's a very hard thing to hold in your head at once. And I think a lot of people don't have good answers to that.

Martin Casado

Can I just test one thing, which may not make sense, but I want to say, I mean, isn't there an argument, though, that like the distribution of the real world is different than the digital world, right? It's heavy-tailed. There's a lot of exceptions. We don't have all the data. And I mean, couldn't…

Diogo Almeida

I don't entirely buy the data argument, in my opinion. I do believe that there's a long tail for sure. Like that, that would be kind of crazy to deny. And I don't think that in my like canary in the coal mine situation, we need to automate that long tail. Like I think that I think we need to be…

Martin Casado

Yeah, no, I remember this is from the Pearl days. Yeah, yeah.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from a16z Podcast