The host had been describing intergovernmental coordination as a brake on the AI arms race, and relaying the objection that such constraints are somehow against evolution. Szathmáry — an evolutionary biologist who has just argued that evolvability, not consciousness, is the real AI danger — answers with the biology. Every jump to a higher level of organisation works precisely by suppressing evolution at the level below it.
…know, the players, the human players, are not a good guarantee that such an agreement will ever come about. What I'm seeing is everybody's racing like crazy, right? Because we have to compete with the Chinese, we have to compete with the Americans. I mean, you know, this is how things can crash.
…evil government thing that's going to be a tyrant. That's against evolution. I would say evolution. Expands its powers and then seems to find ways to bound them. Humans got big brains, but in order to participate in society, we have to stop being like murderous and inherently individually selfish.
A higher level organization always suppresses the evolutionary potential at the lower level. Otherwise, we would have cancer all the time.
Of course, of course. But then it opens up potential at the new vanguard, which is even greater. And so
as Stu Kaufman would say, there are enabling constraints.
Yes, yes, yes, yes. So Spinoza has this idea of potestas, which is different. That potentia is the expansion of powers. Potestas is the selective bounding of them. If done well, potestas increases the net potential of the system. And so enabling constraints is a beautiful way to put it. I'll have…