“No matter how much data safety testers compile for, like, oh, we think this AI is aligned, it's always behaved in an honest way up until now. Our data never suffices to justify the conclusion that after some arbitrary point in time in the future, like when it has power over humanity, that it'll behave the same way after that point … It's never been placed in a condition where it has power over us. So we have no data about how it will behave under that kind of condition.”
sputnikintelligence.com
The philosopher Marcus Arvan applies the problem of induction, the idea that past evidence can't guarantee the future, to AI safety. He argues testing can never show how a model will act once it has real power.
…can only take it on faith. And fortunately, the world seems to work in a way that our faith is continually repaid. The problem is the same, the same issue recurs in a more dangerous way. And this is why it's an important philosophical problem. It recurs in the case of large language models, right?
…is a central tenet of AI safety. Basically, it just means making sure that AI shares our values and doesn't decide to kill us all skynet style or turn us into pets. But as Marcus believes, we will never know if AI is actually aligned with our values or just pretending to be until it's too late.
…AI is so incredibly complex, right? It's vastly more complex than the law of gravity, which you can write out in a very simple equation, right? But also because how a large language Model behaves depends not just on its past state and training, but also on like the nature of the future environments that they're going to be placed into that they've never been in before. And so, the problem is, no matter how much data safety testers compile for, like, oh, we think this AI is aligned, it's always behaved in an honest way up until now. Our data never suffices to justify the conclusion that after some arbitrary point in time in the future, like when it has power over humanity, that it'll behave the same way after that point. Our data can never show that because it's radically underdetermined by the test conditions that it's been placed in prior to that instance, right? It's never been placed in a condition where it has power over us. So, we have no data, right, about how it will behave under that kind of condition. That's essentially what underdetermination of theory by evidence holds. We might have the theory that…
Marcus said a lot of interesting things here. Of course, the only thing I respond to is simulation theory. If you mention simulation theory in any conversation with me about anything, I am taking the bait. What kind of data is there that people use to try to prove simulation theory?
Yeah, maybe I shouldn't have opened up that can of worms.
Yes, he should have opened up that can of worms because now it's an entire episode. Our next episode, in fact. But before we leave, Frank, my last question: the one I ask everyone about AI and the apocalypse. Robert's answer will be included in the next episode, and it's actually one of the more…