“A higher level organization always suppresses the evolutionary potential at the lower level. Otherwise, we would have cancer all the time.” Eörs Szathmáry · The Trajectory

for AI, but you know, the players, the human players, are not a good guarantee that such an agreement will ever come about. What I'm seeing is everybody's racing like crazy, right? Because we have to compete with the Chinese, we have to compete with the Americans. I mean, you know, this is how things can crash.

Yes. Well, so, so, look, I'm completely with you. We've had entire series on this show with Benjio and Stuart Russell and former senators and high-level people from the Department of Defense all talking about. And I've been out to Shanghai. I've talked to the guy who's running sort of dean of Tsinghua, Zhuilan, all talking about different kinds of knitted intergovernmental coordination ideas that could potentially put up buffers against arms race acceleration dynamics. Because I think, and some people say, well, that's against evolution. You're trying to do some evil government thing that's going to be a tyrant. That's against evolution. I would say evolution. Expands its powers and then seems to find ways to bound them. Humans got big brains, but in order to participate in society, we have to stop being like murderous and inherently individually selfish.

Of course, of course. But then it opens up potential at the new vanguard, which is even greater. And so

as Stu Kaufman would say, there are enabling constraints.

Yes, yes, yes, yes. So Spinoza has this idea of potestas, which is different. That potentia is the expansion of powers. Potestas is the selective bounding of them. If done well, potestas increases the net potential of the system. And so enabling constraints is a beautiful way to put it. I'll have to tell Kaufman that you quoted him there. But yeah, I think we absolutely need that. And on this show, and in many places in the physical world, the United Nations, the OECD, places where I'm involved, I'm actively advocating for these things and know many of the people that are pushing this kind of intergovernmental dialogue forward. So I'm with you there. Let's say we get there. And I know it's going to be challenging, but let us say that we arrive there. We have an imperfect, it's always imperfect, some imperfect kind of coordination between the US and China. And now we got these really powerful tools. From there, if we wanted to be part of the greater process, a participant in change, like you had said, transformation or attenuation, it's one or the other. If we want to think about how to positively transform, what do we do? Like we prevent the runaway AI for at least a little while through some kind of coordination. What for you would be next steps? I'd love to kind of be able to close on some of your thoughts here around what you would hope humanity could do should we get to that first stage.