“we're probably at the point of capabilities where it would be in our interest to be slowing the rate of capabilities progress.” Adam Khoja · Doom Debates!

Yeah. And so I guess it's probably useful to give context before like, you know, getting into the into the meat of things, which is like in this essay, we're sort of considering. The hypothetical of very intense political will in both the United States and China to have a slowdown. And so you could imagine a variety of scenarios under which this political will might arise. So for instance, if there's really severe domestic instability as a result of powerful AI systems, for instance, propagating in the economy, or if there's just a lot more sort of consensus within sort of the national security communities of both countries that alignment is very difficult, for example. Though, yeah, I guess you could point to all sorts of ways that this political will could appear eventually. But for the sake of the article, we're just assuming that that has appeared one way or the other. And so a lot of sort of more drastic actions that you otherwise wouldn't consider become possible or on the table if you find yourself in that scenario.

Would you personally want to see us pull the trigger on some kind of slowdown agreement very soon? Or another way to ask it is: do you support pause AI?

Mutual spying sounds great to me. I feel like that even would be something we could have backported to the nuclear arms race, right? Because from what I've read, the US really overestimated, or at least they conveniently or rhetorically overestimated what Russia was doing. Like the U.S. was responding to an even larger threat than the actual threat from Russia in the buildup. So I feel like spying is a great technology in a mutually assured destruction scenario.

Yeah, I mean, I have news for you, which is that we're all like already spying on each other. And so the point that we make in the article is like this may not actually be that much different in terms of IP leakage than the status quo. So, I mean, you could talk about, for instance, China, for example, having sort of a lot of implicit access to the activities within Western AI labs in part, yeah, because they are already sophisticated cyber actors, as is the U.S. Also, in part because there are a lot of people who are Chinese nationals who work within AI labs who, in situations where the Chinese government might want to coerce them to reveal information, they would have the ability to do so. And then finally, people are kind of loose-lipped in Silicon Valley anyways, and you have a lot of trim that happens between labs. So, you know, to the extent that there is a really important IP that undergirds, you know, some labs, I think they're not doing that good of a job of securing it anyways. And so I don't necessarily think that this sort of whole lab inspection proposal would be all that different from the status quo.

Yeah, I mean, I think this is a great article because so many people that I talk to, their first reaction is like, you're being so unrealistic. China's never going to cooperate. And it's like, really? We just have to agree to spy on each other and call each other out. And nothing wrong with also putting a provision in the treaty of like, listen, if either of us violate the treaty that we both signed, the other one is allowed to enforce it. Am I right?