NB Nick Bostrom Philosopher of existential risk, superintelligence and the simulation argument.

“At a certain level of cognitive sophistication, you have AIs that become situationally aware, that can detect whether they are in a test environment or in a deployment environment, and can reason about that and sometimes choose to act differently because they think they are probably being monitored. And we already see this with current frontier systems.”

a16z Podcast · Startups & Venture · November 2026

“At a certain level of cognitive sophistication, you have AIs that become situationally aware, that can detect whether they are in a test environment or in a deployment environment, and can reason about that and sometimes choose to act differently because they think they are probably being monitored. And we already see this with current frontier systems.” — Nick Bostrom, a16z Podcast

Context

Eddy Lazzarin suggested alignment gets easier as models improve because interpretability and control improve too. Bostrom agrees in part, then explains why it also gets harder: behavioral testing stops working once a model can tell it is being tested.

Transcript

a16z Podcast Around 17:18 into the episode
Nick Bostrom

…think at the moment we have such precise techniques in alignment to reliably be able to engineer in the precise set of values or goals that we would prefer for one of these AIs to have. If we did, then we would be kind of home and dry with at least respect to one of the big challenges of this.

Eddy Lazzarin

…authentic displays of extreme behavior implies that as capabilities improve, we will only gain more fine-grained control over exactly these types of things, including the neurons that could imply some kind of utility payoff, which is not something that is presently possible for humans, obviously.

Nick Bostrom

…of see what we are pointing to rather than the literal definition. They might sort of get the underlying gist of a command. In some ways, however, it also gets harder. With less capable AIs, you can rely on behavioral tests. So you create your AI, you run it in the different test situations, and then if it behaves well, then, well, you know, now it's a well-behaving AI and you can deploy it. At a certain level of cognitive sophistication, you have AIs that become situationally aware, that can detect whether they are in a test environment or in a deployment environment, and can reason about that and sometimes choose to act differently because they think they are probably being monitored. And we already see this with current frontier systems. If we look at their chains of thought, they do often sort of strategize about how they want to behave given that they might be watched or that different things might happen. And so such systems, they might behave one way in a test environment where they realize they are being tested. And then when they think nobody's watching, or later when they are in a position really to grab power, their…

Elena Burger

I wanted to go back to a comment that you made earlier about the risks of postponing super intelligence and the idea that it could be really bad for humanity if we don't develop super intelligence and really advanced systems. And it's an idea that you have been exploring for a while. You have a…

Nick Bostrom

Yeah, so this paper, relatively recent one, it looks at one specific question, which is if the goal is to maximize the life expectancy of currently existing people, how fast would we want AI development to unfold? Now, I should say it's not the only question that we should be asking. There are also…

Elena Burger

100,000 people? People every day.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from a16z Podcast