Dwarkesh Patel
Hosts the Dwarkesh Podcast, a long-form interview show on AI progress, timelines, and the people building frontier models, along with economists and historians.
“When I learn how an LLM works and how simple RLVR is as an algorithm, I'm sort of stunned by the kinds of things it can do — that it can learn to build very complicated code repositories simply from getting a yes or no. And here, I feel like AlphaGo actually seems less impressive in retrospect the more you understand it — because you're putting in a lot of bias, you're telling it how to titrate exploration as things go on, you're building this very explicit tree search for it.”
Dwarkesh Podcast · AI Research & Frontier Labs · May 2026
Context
His own contrarian take, put to Eric Jang after Jang had spent the episode explaining AlphaGo's machinery in detail. The claim is that understanding moves the two systems in opposite directions — LLM training gets more impressive the simpler you realise it is, AlphaGo less. Jang's first words in reply: 'I personally disagree.'
Transcript
…be interesting if they're basically kind of we've just talked about how they're kind of making similar predictions or they should be in line with each other. And so I'd be curious if like actually, yeah, you just halving the amount of compute you got to do by keeping in the same network. Right.
…know, really resolve that. But intuitively, yes, they share a lot of representations. So, and as we mentioned, there is a sort of like your policy network and your value network when doing evaluation should kind of agree, right? So there really should be this sort of consistency between them. Yeah.
Tell me if this is the wrong way to think about it. I feel like when I learn how an LLM works and how simple RLVR is, at least as an algorithm, how simple it is, I'm sort of stunned by the kinds of things it can do, that it can learn how to build very complicated code repositories and whatever simply from getting like a yes, no. And here, I feel like if you understand it more deeply of like just predicting MCTS, and it actually seems AlphaGo seems less impressive in retrospect the more you understand it, because you're like, oh, you're putting in a lot of bias by just saying how much you'd you're like telling it how we should titrate exploration as things go on. You're building this very explicit tree search for it. And so I don't know if you show that intuition where it actually, the more you understand and the less impressive the accomplishment in 2017 seems.
I personally disagree. I think they're profound for different reasons. And I don't understand the LMRL like enough to kind of comment on your podcast about it. But I think AlphaGo, so yeah, why is it a profound accomplishment? I think maybe it's worth stepping back a little bit and just like, it is…
That is a very interesting insight that a lot of problems which are proven to be NP hard, like I don't know if Go is proven to be NP hard, but protein folding, et cetera, have been like neural networks can solve them because they're NP hard in the worst case, but we're not dealing with the worst.…
I think that the kind of question we should be asking ourselves is like, we've been formulating solutions to NP hard problems as a kind of worst case complexity. And I wouldn't say this solves Go, right? It doesn't give us an exact solution of the optimum. But in practice, it is extremely useful.…
More from Dwarkesh Podcast
