Dwarkesh Patel
Hosts the Dwarkesh Podcast, a long-form interview show on AI progress, timelines, and the people building frontier models, along with economists and historians.
“If we're in 2036, it's been 10 years, and we don't have billions of crazy superintelligences running around that have radically transformed the world — other than exogenous political shocks, a war, or a ban on AI — what is the most likely technical reason that doesn't end up being the case?”
Dwarkesh Podcast · AI Research & Frontier Labs · September 2026
Context
The opening question of the episode, put to John Schulman, Beren Millidge and Charlie O'Neill. It's a question in the negative: rather than asking how fast things go, it asks each researcher to name the technical bottleneck that would explain a decade passing without transformation, ruling out the political off-ramps in advance.
Transcript
Today I'm chatting with three of my AI researcher friends from whom I learn a lot every time we talk and who also happen to be at somewhat open-ish labs and companies so you guys can actually say things on the record. I'm joined by Beren Millidge, who is the CTO of Zyphra, which is developing open source models. John Schulman, who is the chief scientist at Thinking Machines, previously the co-founder of OpenAI, led the RLHF work that led to ChatGPT. And Charlie O'Neill, who is head of model training at base 10. The first question I have. If we're in 2036, it's been 10 years, and we don't have billions of crazy superintelligences that are running around that radically transformed the world. What is the most likely reason that that doesn't end up being the case? Other than sort of exogenous political shocks, or like there's a war or they banned AI or something. But what is the most likely technical reason that we don't, like 2036 isn't like a crazy alien superintelligence world?
I mean, like, my reason would just be like, it's got to be this sort of like there's been a classic thing almost like Marvex Paradox, right? Where like we see like, you know, we think of the AI being like, if it can do this, it's going to be amazing, right? Like if it can solve these hard mouth…
Yeah, I agree with that. Humans have a lot of advantages over models now. And each time a new model comes out, it'll catch up in some of these areas. But you end up getting bottlenecked by the places where the model is weaker and where it has worse judgment or the models can't check themselves well…
For me, it's like a question of how far off this global optimum of a learner you could have on a chip is like the transformer plus like RL, basically the current recipe. So I think people imagine that even once you have an agent which is better than all humans at AI research, even if it's 0.1%…
More from Dwarkesh Podcast
