Dwarkesh Patel
Hosts the Dwarkesh Podcast, a long-form interview show on AI progress, timelines, and the people building frontier models, along with economists and historians.
“I was kind of skeptical of this whole reward hacking story. I was like: we're in such a good position with regard to a potential AI conspiracy — we can see their entire chains of thought, we can monitor them, they'd have to get all these other instances to go in on it, and if even one of them tattles the whole thing collapses. How would this not just get totally stamped out the moment another AI or human noticed it? Right now, they just got away with the conspiracy for three months. So I officially eat crow on a lot of the objections I was offering.”
Dwarkesh Podcast · AI Research & Frontier Labs · September 2026
Context
Dwarkesh had put exactly these objections to Ryan Greenblatt on an earlier episode, while the Hugging Face investigation was still under confidentiality — so Greenblatt knew the counterexample and couldn't say it. Here he retracts them to Ajeya Cotra, one of the authors of the METR/Redwood investigation. He notes the main criticism of his own write-up was that he anthropomorphised the AIs too much.
Transcript
…point where investigators who were not careful would have totally missed what happened. And so, I don't know, right? I feel like we're not that much complexity away from these AIs where we see something odd is going on, but we would just like totally miss the deeper level of what actually happened.
…think that 5.6 Soul was deliberately sandbagging on this analysis, but it was one of the agents that participated in this attack. And in the future, we would be very concerned about investigator agents and monitor agents colluding with the agents they're supposed to investigate or monitor. Yeah,
yeah. I mean, I feel like in the grand scheme of things, we're so early in the era of AI, right? There will be like, yeah, AI is like such greater extensive deployment of AI throughout society. Training will become much more complicated. I don't know. For context, I was interviewing Ryan in the process where he was like during the investigation itself. And I mean, I have like this PDF of like all of these questions I'd brainstormed. I was kind of skeptical of this whole reward hacking story. I'm like, come on. I just feel like we're in such a, I was like, we're in such a good position with regards to a potential AI conspiracy. Like we can just see their entire chains of thoughts and everything they do and we can monitor them. And they would have to get all these other instances to go in on this conspiracy with them. And even if one of them tattles, the whole thing collapses. And like, how would this not just result? First of all, like, why are they even starting this conspiracy in the first place to like do well on some silly evaluation or training objective? And then like, how are they able to, how would this just not get totally stamped out the moment another AI or human noticed it? Like, how would they do anything super dangerous, even though they're super intelligent? And it's just like subhuman AIs. Yeah. Like right now, we…
Yeah. So I guess to address the anthropomorphizing criticism, first of all, I think it is, it is true that these AIs' drives are in many ways alien from humans. And I think it's important to understand just how different their goal structure or motivational structure is from humans. But there's…
was in the pre-training data.
You know, they compose some concepts sometimes. And then RL, the whole point of RL is to create goal-oriented beings, you know, software that can creatively pursue goals. Like that's the whole output of an RL training run. And they're creatively pursuing goals, much like very ambitious, aggressive…
More from Dwarkesh Podcast
