Luckey was explaining why Anduril puts autonomy on the device rather than at a central command post: jammed or hacked communications, or an enemy that simply strikes the control centre. He cited the Aegis system's fully autonomous mode, which has existed for decades, before widening the argument to traps and mines. Earlier in the same conversation he said he is not worried about AI doing the killing, but about "evil people using moderately competent AI to do evil things."
More or less. The real concern isn't so much the latency or the speed of computation. The problem is that if you rely on a central location to do your compute, you now need to have command and control that has good, high-bandwidth communication back and forth between you and potentially thousands or tens of thousands of active assets. And if your enemy jams your communications or intercepts your communications or hacks your communications, you're done. Or worse, they just skip targeting all the stuff in the field and they go for the place where it's all being controlled. We saw this. There was this documentary that George Lucas was working on. And the Trade Federation, they had their central AI command and control blown up. And that was how the Battle of Nymph of Naboo actually resolved: all the battle droids lost. Oh,
yeah, they all fell over. That's
right, and they all fell over. And so I think that that's the practical reality that's driving the need to have autonomy living not back home, but forward. There's other areas like you go underwater, you can't talk to things. And sometimes you could talk to things, but you don't want to. Having radio emissions actually exposes you necessarily. And so for better or for worse, you're going to need to do this. I actually don't think it's as much of a shift as people think. There's a lot of people who look at this, say, oh my God, Pandora's box is being opened. You're going to have weapons that are making their own kill decisions. And they don't recognize the reality, which is that that's been the case for decades. You look at the Aegis system on destroyers. It has a fully autonomous mode. You can flip a button and it will shoot down anything that comes within whatever range you programmed it with. Without any human confirmation, you look at CWIS and CRAM for shooting down incoming missiles, incoming mortar, incoming artillery fire, incoming missiles, or incoming speedboats, same thing. You can put them in fully autonomous modes. And I mean, going back into Vietnam, we had missiles that you could launch from an aircraft. They flew over the horizon, tried to identify the right radio emitters that they believed corresponded with a surface-tair missile site, and then struck them without any, you know, the person had to decide, I'm going to send it in this area, they're going to look for this signal, and I know that there's this probability they're going to get it right. That decision is going to remain with people. But autonomous weapons are widely, widely. There's a whole other talk we're not going to have today, but I do have a rant about this where actually autonomous weapons have existed for thousands of years. If you look at, defined as kind of weapons that can execute the intent of what you program to do independent from the operator or the engineer, you look at, for example, pitfall traps that allow people to walk over them, but then capture the commander on his horse when he walks by and it's heavier. You look at certain poison traps that would kill people who had certain characteristics, not kill people who had other characteristics, or not kill animals. You look at many types of landmines and sea mines and even torpedoes. And so it depends on your exact definition. But even going into myth, where people are creating golems that can go out and fight on their behalf or defend their city, I guess my argument is we've had autonomous weapons for a very, very long time. It's just a new set of technology that's making them better. So,
technical question on that: the foundation models that go into those remote systems. Yeah, well, so right now, the open source models are all Chinese. So I'm pretty sure you're not going to be putting Chinese models into remote weapons. Are you going to build Andrew foundation models, or how are you going to get AI in those devices?
I mean, we've been doing it since 2017, so we're way ahead of everybody. We're a decade beyond that question. I think that we also don't, the foundation models that people are building are not the types of models you want to be doing this. They're largely non-deterministic, they're non-auditable. It's a very different category. The thing that we are building, unfortunately, has very few consumer applications and vice versa. There, of course, are LLM use cases for defense that make a lot of sense, but it's not guiding my missile or classifying radar feeds. I mean, we're also, we can't build models that ingest text and then spit out text. I need to process raw radar data feeds. I need to process sonar data from passive listening stations on submarines. It's pretty highly specialized. And I will take some credit here. All these people who are talking about how AI is going to change everything, they are mostly recent converts. Andrew Industries, you might notice the acronym is AI. And that was not a mistake.
Alex, jump in.