“It's like explaining our motivations to a rabbit, like explain to a rabbit why we built a skyscraper. It doesn't make any sense, right? Because it's just not the universe they're in. And so what I worry is that AI is going to become to us what we are to animals.”

trick. It's just like, it's like unaligned humans. And then like, because a weird thing about the world, or this is, this is the thing Dario talks about. So I should say that, like, you know, this is like a little bit of some Darioism going on here. Is like it used to be the case that like the more intelligent a person is, the more aligned they are with humanity's goals generally. This is like why we don't have that many terrorists is because like the people who are smart enough to pull off a biological terrorism attack are also the people that are smart enough to realize why that's bad. And those things were correlated. And so one of the like dangers of AI stuff is by like giving intelligence to people who don't have it. And this is where this gets like a little dicey because like you start being like, what is intelligence? How does that matter? And you already measure this and whatever. But like the point is that like stupid people who don't understand that doing biological terrorism is bad, having the ability to synthesize new viruses, it gives this power to people. Previously those things were connected and that was somewhat it was keeping us safe. And that's like a tricky part about this situation, basically. Like, cause people, people using Claude for everything. Like there's, there's some, like we talk about joking about like, oh, you wrote your PR description using a chatbot. Like that's bad. Like people in the CCP were using Claud to produce reports about like the quarterly regional crop growth or whatever and like sending that off to Anthropic and they were like seeing copies of that or whatever. And it's like, we have state secrets because like the government officials are using Claude to help them prep stuff. And it's like, you know, just like let alone like actual like Houthis using Claude to like test rocket drones and, you know, or whatever, like, and like, you know, like whatever. These are obviously politically charged examples. But like the point is just that like people are going to use this for all kinds of stuff that's way crazier than like, it did my homework for me or whatever. And so like we do have to think about the consequences of humans doing bad things with this, even if the AI can be aligned in a way of like doing what we told it to. That's not always actually like the right way of thinking about the problem.

Which is also why it's important to not, because I think the thing that I want to push back against is the like people are more worried about what the machines will do than they are about what other people will do. And it's like, that is, it's, it's sensationalized because it's like, oh my God, sci-fi tells us this. But I'm like, yeah, but history says whenever you have a very large expansion in the information that an individual has access to and they can expand their knowledge, like there's good things that come from that and bad things that come from that. And we need to like be careful with how we do that because we don't want to restrict knowledge necessarily. But you also don't want people, as you say, that like don't understand why terrorism is bad to go do terrorism. Right. So it's like a balance, but it's a balance that we've been handling for a very long time now. So we just need to figure out how to also handle it this for this iteration, but we're not doing that because we're too obsessed with the sci-fi aspect, which is what I think is a dangerous thing, right? We're not, I mean, we are like a lot of, there are a bunch of people in tech that are having the conversation around like, hey, we also have to worry about humans using this poorly. And like we should think about how we restrict this for humans so they can't use it poorly in addition to having misaligned AI. But I don't think that that conversation is getting out into the rest of the ecosystem. I think there's just a lot of people who are really. Really, really worried about Skynet, which on my list of things I'm worried about is very, very low because, like, this is not every time I, every time I try to use AI really heavily, I'm like, oh, right, this thing is, it's not magic. It is not magic. And so I think like, I, I, I want us to make space for these other conversations so we can actually understand the actual harm that might come from these things, which is probably not it becoming a sentient super intelligence that enslaves all of us.

I feel like it flows in the other direction too. There's a lot of things that animals do that we do not understand nor comprehend well. Yeah, yeah. And it's like, so it flows in that direction. But I also think like, I don't know. I think part of what you said there just like reminded me, like there's this, I think when we think about intelligence, we forget that like intelligence is not about the individual necessarily. Like you, like all of us obviously have like a lot of intelligence. Like we're having this conversation, but like the real intelligence of the world is like the collective. Yes. Like us as a collective right here is an intelligence and it's much greater than any of us individually is. And that's the same thing of like all humans collectively and then like the different agents collectively as well. And I do think that in that case, we can start to understand why things that are smarter than us, as in the collective intelligence versus our individual intelligence, do things and behave in certain ways. And I do think that like, yes, we have accidentally extincted fishes because we didn't know different species because we did not know that. You know, that we were doing harm to them, or because, yeah, or we didn't know, or we didn't care, but at the same time, like those factories also like killed lots of humans or gave them lots of asthma or did other bad things to like actual people that like can comprehend why this is happening. But I do think that part of it is that, in a way, a lot of people do not comprehend why things happen because they can't understand the nuance of it. Like, this is this is why I'm not as worried about like AI specifically killing us all or like having like a p-doom there. Because, in order for AI to do it, it wouldn't just be AI, it'd be AI plus a thousand other things, right? There's like so many different things. Like, if you think about like why is it that America has done a lot of the bad things that America has done, it's because like not just, oh, there were some evil people in power, but there was also a lot of other people with a lot of different motivations that did or did not do something that probably they should have done to avoid the problem that collectively added up to bad thing happening, right? So, it's like, you know, I guess an example that's just like, why did 9-11 happen? It's like, well, there's a lot of reasons why it happened. There were intelligence failures. There were the banks putting all of their data into buildings for some reason. There were like a ton of stuff. There was like us being in the Middle East and pissing people off. It's like, there's a ton of different reasons why that specific thing happened. And we can't just boil it down to like, oh, people mad at us, right? There's a lot of reasons why that particular thing happened in that particular way. And that's the case for like all of this stuff. So like, if there is a future in which we do get extincted, AI might play a part in it, but there were also a lot of other things that led to the situation where we wound up getting maybe near extinction. Like it's going to take a long time to kill off like eight to ten billion people on an entire planet. It's definitely possible to do, but it would take a lot of effort to actually get rid of all of them. Yeah, yeah. So it's like, I think it's, it's easy to conceptualize the worry, but I think we have to like play out like, well, in what way is this going to happen? And then what way can we figure out to make sure that that doesn't happen? And part of it is AI alignment. But I also think a lot of it is like the things we need to be doing anyway to make society suck less and actually make progress again. Because I think society has stagnated a lot of ways making progress. And I think a lot of that is because people just don't understand how anything works. Like we have a lot of literacy crisis. We have a lot of comprehension crisis. So it's like we need people to understand how things work so we can fix the problems because yes, AI might cause a problem, but there are also all of these other threats that we have to worry about that could also lead to the extinction of humans. So it's like when I put everything into the machine, it's like, yeah, AI is a problem, but there's all these other problems that we also need to go and figure out how to solve. And we shouldn't be, it's not helpful to be disproportionately worried about AI because there's all this other stuff that we, that is much more prescient that we should be worried about. Like, I don't know, we should, and there are a lot of people that are pissed off about this, but like Flock and ALPRs, like that needs to go. Like we need to not be doing that. That's really bad for us and harmful as a society. But like the ambition to drive forward is like really making it so that a bunch of people are doing bad stuff around all of this. And it's like, okay, well, that's a problem we need to like figure out how to solve. And there's a bunch of little problems like that that need to, we need to figure out how to solve them so that we don't wind up declining into, I don't like the word, but like fascism or some sort of like other terrible situation or increasing the bad things that we're already doing, society. So I think it's for me, it's less that it is like, I'm worried, it's less that AI won't kill us all. It's that there's a lot of things that might kill us all. And I just don't put AI very high on that list. Hello, viewers/listener. Thank you so much for listening to this episode of Fall Through. I just wanted to let you know that this is the regular edition of Fall Through. And we actually have an expanded edition where Matt Steve and I get into even more depth on this whole topic. If you want to listen to that expanded edition, you should become a supporter by heading over to fallthrough.fm slash subscribe and signing up today. Not only do you get access to expanded editions of Fall Through, you also get a bonus discussion in each episode of our sister show Continue Good Proposals Edition, and you get access to higher quality audio-only editions. In addition to that, you also help keep the podcast sustainable. Again, if you want to sign up to become a supporter, please head on over to fallthrough.fm slash subscribe and sign up today. But for now, let's get back to the show. It's, you know, once again, as with everything, it's a big mess. It's a big mess that we have to like sit down and figure out how to work our way through. But my point is like we need to talk about it all with more nuance and not be like, oh no, Skynet's going to come and kill all of us. Because there's, it's just not, it's not helpful. And Like people do actually. They're like, oh my god, Skynet is like, we're so close to it. We're so close to Agent. Like, because I hear people say things like, there's a greater than 10% chance that in the next decade, we will all be extinct. Which I think that's that's a statement that I'm like, okay, like no shot. Like, not no shot. Like, I don't think it's a, it's not like tautological. So it's not like definitionally false, but it's like such a low percentage in my mind that I'm like, I mean, killing all humans in a decade. That's a lot of killing you gotta do. Like, that's a, that's a, and like the AI can't even like it can't do things yet. So we got to get to the point where it could do things and then it could massive. That's all that's a very short timeline. So that specifically, I'm like, that that's BS. But the concept of it extinct all of us, I think, is worthy of thought. I don't know. Steve, do you think that the AIs can kill us all within the next decade?

I don't know. I think it's not impossible. I think the problem is the curve. Everything is happening faster and faster and faster. Everything is going. There's more and more everything all the time. And so, like, true. A year and a half ago, if you told me that I would like give an AI access to my emails, I would be like, what the are you talking about? And now I'm like, hey, Instinct, please, please go search through all my find me some things, please. Like, so, you know, just like at the rate at which I have like watched this happen is like very it's just you know, and you don't know what part of the curve you're gonna end up at exactly, but like it's uh I don't know. I just I just like see things going like this. It's like, what's going on? As much as

I think that too, I have a lot of, I think there's more of a chance that we will pivot in a hard direction that makes all of this irrelevant than that we will stay on the same curve and continually like there is a curve. Like things are definitely accelerating. Like we are all socializing to like be like, oh yes, AI is like, sure, I'll give it my email. Like now you can use plaid to like hook up your chat GPT to all your finances, which is also a thing. I'm like, I don't, I don't like that. I did.