After describing how AlphaZero improved by playing itself and why that works in verifiable domains like maths and coding, Demis separates today's recursive self-improvement, with humans in the loop, from a closed-loop stage he thinks labs should approach with great caution.
…or less than 12 hours, it would start from random play in the morning to, you know, you'd have your coffee break at 11 a.m. and I could still just about beat it. Then by lunchtime, it was grandmaster level. And then by tea time, it was better than any chess-playing entity. Tea time is dinner
time for the US company. Yeah, computer time
…how to do it without the human reviewer process in the whole procedure. And that will obviously be faster, but I think is also a very dangerous way to get to full AGI because we're likely, for two reasons, it's sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know how do you control, keep control of such a system, such a process. And then the end result will likely be quite alien. It's going to be like an alien intelligence. There's no reason you to think it would be sort of in the similar kind of mind space that humans are in. So that will make it very hard for it to understand for us to understand it and maybe for it to understand us. So I think it's, you know, we should be very cautious as we get closer towards that closed loop form of RSI. And I think all the AI researchers sort of know that, but there is a sort…
So presume that the incentives to be in a race continue. And, you know, you and I, some people I know this, you know, have tried to figure out if there's various ways you can get global collaboration to have less. You've dedicated a huge amount of energy to that. So thank you. It's a hard game, as…
Well, what we're doing, and I think other leading labs are trying to do, is you've got to, so there's more prosaic things like make sure cyber defense is accelerating faster than cyber attack and making sure you give access to trusted cyber defenders first on maybe slightly fine-tuned versions of…
Should there be ways that the labs are encouraged, required, permitted to collaborate on this? And if so, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology, you know, mutual accountability.
