“The issue is going from AIs that are safe because if they were trying to take over the world, they would fail to AIs that have to be safe because if they tried to take over the world, they would succeed.”
Odd Lots · Money & Markets · October 2026
Soares says the labs are always fighting the last war: each new incident gets new monitoring and safeguards, and then a smarter model fails in a new way. Trial and error is how science normally works. He argues it stops working at this one threshold.
Transcript
…they got smarter to the point that they could sort of break out, make an unsanctioned message board, and hack their way out of a sandbox that was supposed to have no internet access without seeming to really find it very difficult. Right. And what we're sort of seeing there is everyone's always
fighting the last war. Yeah. And then, you know, the new war happens and it sort of, you know, smacks them across the face
…new safeguards, and they're going to make the AI smarter. And it's going to create a new sort of problem. And this is the usual way that scientific progress goes is by trial and error, where, you know, when you put a new type of system in a new type of environment it's never been before, it breaks down in some new way. And then you say, oops, and you diagnose it and you fix it and you move on. The issue is going from AIs that are safe because if they were trying to take over the world, they would fail to AIs that have to be safe because if they tried to take over the world, they would succeed. That's one of those phase changes. That's one of those like, oh, you're putting the AI in a different sort of environment where it never was before. The difference between an AI that like can really see a way to become self-sufficient, not engage with humanity, and the sort of AI that's just being told that in training, that's one of those like, oh, we like, this is a different sort of AI…
of AI system, different sort of scenario. You're going to have a new problem that crops up. Except this time, when the new problem crops up, it's going to be able to turn you off before you turn it off. And so this usual process of like, oh, we're going to respond when the new type of problem…
ChatGPT Work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials…
local support and a broad range of resources to more than 7 million customers. With a deep understanding of your day-to-day needs, they provide products and guidance built to help you thrive. Right now, earn $500 when you open a new Chase Business Complete Checking account for new business checking…
More from Odd Lots
