Fallthrough · Hands-On Engineering Podcasts · October 2026
Klabnik was explaining why he puts the chance of AI catastrophe above zero. He isn't picturing a hostile machine. He points out that humans have wiped out species by accident while pursuing their own goals, and worries that a system with its own goals and enough control over the world could do the same to us.
…than like, it did my homework for me or whatever. And so like we do have to think about the consequences of humans doing bad things with this, even if the AI can be aligned in a way of like doing what we told it to. That's not always actually like the right way of thinking about the problem.
…this thing is, it's not magic. It is not magic. And so I think like, I, I, I want us to make space for these other conversations so we can actually understand the actual harm that might come from these things, which is probably not it becoming a sentient super intelligence that enslaves all of us.
This might be where my brain is just too cooked because, like, I both agree with you, but also I think it's good that people are worried about Skynet now rather than later, because you got to be worried about Skynet now rather than later, actually. And the reason for that is like, there's kind of the two sides of the coin. So, one reason why my P Doom is non-zero is because humans might kill us all with AI. And I think that's like a viable situation. So, like, sure, we already talked about that. That's like one reason. But the other reason is when I think about extinction and how it has happened historically speaking, like in the vast majority of cases, like maybe not the vast majority, because while humans are present, like we have caused species to go extinct by accident. Like, we did not intend for some fish in some river that only lived in that river to like go extinct and there's no more of that specific kind of fish anymore when we built a factory and polluted the water. Like we were just pursuing our own goals and we were doing stuff and…
I feel like it flows in the other direction too. There's a lot of things that animals do that we do not understand nor comprehend well. Yeah, yeah. And it's like, so it flows in that direction. But I also think like, I don't know. I think part of what you said there just like reminded me, like…
I don't know. I think it's not impossible. I think the problem is the curve. Everything is happening faster and faster and faster. Everything is going. There's more and more everything all the time. And so, like, true. A year and a half ago, if you told me that I would like give an AI access to my…
I think that too, I have a lot of, I think there's more of a chance that we will pivot in a hard direction that makes all of this irrelevant than that we will stay on the same curve and continually like there is a curve. Like things are definitely accelerating. Like we are all socializing to like…