Dwarkesh Patel Hosts the Dwarkesh Podcast, a long-form interview show on AI progress, timelines, and the people building frontier models, along with economists and historians.

“Here's a claim I'd make: I don't think there's any environment I could make that's supposed to emulate real economically valuable work — which requires interacting with other people and the world — where, if you were in that environment, you would not realize with enough effort that you are in that environment.”

Dwarkesh Podcast · AI Research & Frontier Labs · September 2026

Brown had suggested that sufficiently realistic evaluation environments are how labs could check whether a model is aligned before handing it real power. Dwarkesh's objection is that any environment realistic enough to test economically valuable work is also realistic enough for the subject to figure out it's a test. Brown's counter is that Dwarkesh is not an ASI, and a powerful enough model might build environments that fool other models.

Dwarkesh Podcast · 2026-09-17 Listen to the episode → More from Dwarkesh Patel → More AI Research & Frontier Labs quotes →

Transcript

Dwarkesh Podcast

Dwarkesh Patel

But anyways, how would we know that we're like keep we're doing RSI? Is it, I don't know, are we like on track to build the billions of ASIs that are gonna like aligned or not aligned? How will it like, how will we know that?

Noam Brown

…is a trap and they want to find some other way. It's just like, no, they're just like, oh, I'm in a test environment. Okay. Like, so making an environment that's like realistic enough that it matches, that it's indistinguishable from the real world for them is becoming increasingly more difficult.

Dwarkesh Patel

Here's a claim I'd make. I don't think there's any environment I could make where it's supposed to emulate real economically valuable work, which requires interacting with other people and requires interacting with the world. I don't think there's any such environment I could make where if you were in that environment, you would not realize with enough effort that you are in that environment.

Noam Brown

And you are not ASI, right? But you are a human. So if you have very powerful AI models, maybe they can make environments that could accomplish that.

Dwarkesh Patel

No. It just doesn't seem, especially if they've been relying on the AIs. Are they in on the scheme? I don't know. It just seems like it's like, yeah, this

Noam Brown

is another thing that we want to measure. And there is, I think this is actually one of the strong arguments for not training AIs to be fully cooperative. That if that leads to an increase in basically collaboration when the agents are supposed to have different objectives, then that is a problem.…

Speaker names from our own diarization · position estimated from where the line sits in the episode

More from Dwarkesh Podcast

“In a way, the Aztec model makes more sense because sacrificing your enemy means you'll never see him again, whereas selling your enemy back to his family means he's up for round two and you might face off against the same guy across a future battlefield.” — Si Sheppard, Dwarkesh Podcast

“I wrote a blog post a few months ago about how RL, at least policy gradient RL, is even more inefficient than you might think. The inefficiency one thinks about naively is that you have to roll out a whole trajectory to get any learning signal at all — instead of completing the next word in a sentence, the model has to do two days' worth of work to figure out if it even did the project correctly. If you're doing this through RL, the model would try 'the sky is halcyon' — nope, wrong. 'The sky is told' — nope, wrong. This is a totally untrained model, so you'd have to do this on the order of 100,000 times just to stumble on 'blue' before you get any learning signal at all.” — Dwarkesh Patel, Dwarkesh Podcast