“They showed me their decision-making system. It consisted of six steps, each of which at the time was taken by a human. And the shortest element of time was the human decision — that was the quickest thing. But once they added AI into the system, three elements of that decision-making were given over to AI… and it changed so that the longest part of the cycle ended up being the human judgment, just because everything else got so much quicker.”
Future of Life Institute Podcast · AI Research & Frontier Labs · September 2026
Context
Manson, who has spent years reporting on the Pentagon's Project Maven for her book, is describing what she was shown at the US Army's 18th Airborne Corps. Gus Docker had asked whether human judgment is already a bottleneck in AI-assisted targeting. Her answer is that the military may now read careful human deliberation as the slow step — though, as she notes, it is also the step where the law of armed conflict gets considered.
Transcript
…the military will share that publicly. It's not information I have access to. But we do know that there are hundreds probably of reports of civilian casualties in Iran that have been submitted to CENTCOM because the NGOs say they have submitted to them. And they haven't yet been adjudicated on.
So to what extent is human judgment already a bottleneck? If you, as you mentioned, the system can present thousands of potential targets. So to what extent is the decision-making of lawyers and soldiers and so on already a bottleneck here?
It's a really interesting question. When I went down to the 18th Airborne Corps, they showed me their decision-making system. It consisted of six steps, each of which at the time was taken by a human. And the shortest element of time was the human decision. That was the quickest thing. But once they added AI into the system, three elements of that decision-making, the steps in that cycle, were given over to AI. This was about data collection, data analysis, assessment, those kinds of things. And it changed so that the longest part of the cycle ended up being the human judgment just because everything else got so much quicker. So I think it is at the stage where the decision is taking longer than other parts of the process. If what I learned at 18th Airborne Corps is being carried through to central command, which I don't know. But yes, from a military standpoint, it might be seen as a bottleneck, but it might also be seen as the most critical part of a cycle where the law of armed conflict is considered. There's also another stage, which is battle damage assessment, which is very heavily dependent on getting data back, which the US has always struggled with. And getting those kinds of things right are very important for…
Yeah, maybe you could give some examples here of how the system might fail.
Yes. So some of my reporting for the book focused on the work that the US did for Ukraine. And there, there was a team based in Germany that was using computer vision to try and find objects for the Ukrainians to hit Russian enemy objects. And in the first few days of the war, of Russia's invasion…
When mistakes are made in this system, how easy is it or has it become easier to find out where the mistake was made and ultimately to assign blame? Say, for example, that you have a mistake where a civilian target is bombed as opposed to a military target.
More from Future of Life Institute Podcast
