“We are pulling the trigger on this Russian roulette with the planet. And every shot that doesn't kill us will make us stronger potentially and wealthier and better off. But when do you stop pulling the trigger? At one point, you know that it will be one too much.”
The Generalist · Startups & Venture · October 2026
Tallinn has just said he thinks the current way of building AI is fundamentally unsafe, while granting that today's models are a net positive. His point is that nobody knows which generation will escape control, so each success makes it harder to stop. He argues for stopping now and finding a more controllable approach.
Transcript
I think that it is just fundamentally unsafe. The current paradigm is fundamentally unsafe. Myself, I would have stopped like earlier, which like in retrospect, that's a mistake. I think the current models are net positive. Like you never know what generation we're going to escape control and then you just lose.
Because you sort of think we may have... Yeah, it's not a game we're going to have many chances to learn from.
Exactly, exactly. So we are pulling trigger on this Russian roulette with the planet. And every shot that doesn't kill us will make us stronger potentially and wealthier and better off. But when do you stop pulling the trigger, right? At one point, you know that it will be one too much. So yeah, I would say it's just like stop pulling the trigger right now and figure out what is a better better more. Controllable approach to AI in general. And there have been suggestions now. Yoshio Benjio has this idea of scientist AI that is deliberately trying to tease out agency from AI. So it's like kind of like principled approach, how you can make non-non-agentic AI.
Okay, I haven't read about that. That's sort of the idea of, you know, like a drugged tiger in some way. Yeah,
it's basically like Oracle done properly, where you can ask Oracle. The big problem with like naive Oracle doing naively is that Oracle still has preferences. It prefers giving answers that come true. But if it has just this preference, it is incentivized to make sure that it's going to be asked simple questions, which means that it's incentivized to mess with the world. However, Yoshio's approach, the way I understand it, is that doesn't have that flaw. It basically truly, the Oracle doesn't care what will be done with the answers.
And is this an AI that is explicitly non-agentic, just sort of something you literally visit as an Oracle and ask for advice rather than something that can do things for you?
More from The Generalist
