The host had compared fears about AI to the physicists who worried before the first nuclear test. Bach uses his own change of mind about nuclear weapons to answer, though he adds straight away that a weapon over a few decades is a different case from superintelligence over the very long run, and that the comparison is hard to make and hard to dismiss.
…about superintelligence, not in a Donald Trump renaming sense, but smarter than all of us combined. There's a levels to it. System smarter than you, 10x, 100x, million X. Those are not the same problem. To suggest that you will be able to always stay in control of all of them seems insane. But
…consent before we shut things down? I mean, at what point are you comfortable with putting the Gini back in the bottle if Roman's correct? I mean, your P doom might be irrelevant, but how do you speak for the rest of humanity? Not you personally, but how do we as scientists or as technologists?
There are a number of considerations when we try to compare the nuke situation with the situation of artificial intelligence. And it makes it very difficult to compare even, and it's also difficult to dismiss the comparison out of hand. I, like many people in my generation, Generation X, think that now that I've been wrong about the impact of nuclear weapons. I grew up thinking in the 1980s that we are just a hairbread away from assured destruction and that the risk of nuclear armaments is way too high for humanity. Now I think the adults had been correct. And a generation of students who took to the streets and were very upset about this turned out to be wrong. Nukes have prevented several world wars that would have been happening between World War II and now that would have wrought tremendous destruction on Europe and potentially large parts of Asia and maybe even America. And this basically more than half a century of peace that we had is completely unprecedented for large nations like ours. And I think this was achieved due to nuclear armaments, which have been empirically quite safe. Now, it's very difficult to compare the different timelines. We cannot really sample this. We don't know…
The detritus, 5% electricity, 6%.
Ultimately, the question is: what do we identify as? Do we identify as a kind of ape that is very selfish, or do we identify as intelligence and consciousness in the universe that is currently incarnated as us? And we are handing off the torch to our children, and our children are going to be a…
Roman, when I was walking over to my studio, there's a spider that inhabits this area, and its web is always, you know, I'm about close to six feet tall, and it barely brushed on my hair, and it's always at that height. You know, and the spider's kind of adapted to me and understands me. And, you…