SG Steve Gibson Steve Gibson Software developer and security researcher who founded Gibson Research Corporation, maker of the SpinRite disk maintenance and recovery utility and the ShieldsU...

“For theorem proving, being wrong doesn't really matter. The AI generating the proof does not need to be perfect. Every proof it produces is handed to a proof checker, the kernel of Lean, which will only accept a proof where every single step is valid. Consequently no hallucinated proof is able to slip through. The proof will simply fail to check, and the AI will try again. The model can be wrong a thousand times. It only has to be right once. And the checker tells us when that happens.”

Security Now · Software Engineering & Infra · November 2026

“For theorem proving, being wrong doesn't really matter. The AI generating the proof does not need to be perfect. Every proof it produces is handed to a proof checker, the kernel of Lean, which will only accept a proof where every single step is valid. Consequently no hallucinated proof is able to slip through. The proof will simply fail to check, and the AI will try again. The model can be wrong a thousand times. It only has to be right once. And the checker tells us when that happens.” — Steve Gibson, Security Now

Context

Steve was explaining how Google DeepMind used AI to produce a machine-checked proof of Signal's cryptographic code. He had just raised the standard objection that language models are fluent, confident and sometimes wrong. This is why that flaw stops mattering when a separate checker verifies every step.

Transcript

Security Now Around 2:23:53 into the episode
Steve Gibson

…And then we're going to dig into something. I mean, another, we already talked about AI and this potentially groundbreaking innovation from MIT. Here's another one. The idea that we could actually, finally prove to a mathematical certainty that code is correct and bug-free and does what we want.

Leo Laporte

…about January 8th. Just some of the things coming up. twit.tv slash clubtwit. Help us help you by joining the club. We appreciate everything you do to keep us on the air. Now we continue on, ladies and gentlemen. Join the club, one of us with Security Now. And wow, the case of the provable code.

Steve Gibson

So, yes. Last Tuesday, a week ago, November 10th, Google DeepMind published a research post titled Introducing Alpha Proof Code, a breakthrough in verifying complex software at depth. Okay, so a little bit of background. And this is, I warn you, going to kind of be a conceptual thinking piece here. For anyone who has spent their career around software, what Google is claiming reads almost like science fiction. AI agents, guided by a small team of human experts, produced a true mathematical proof that the actual compiled machine code, because that's what it used, it didn't use source. It used the machine code of Signal's core cryptographic library delivers four specific security guarantees, forward secrecy, authentication, post-compromise security, and handshake secrecy. These were implemented in two parts of Live Signal, the PQXDH handshake, which we talked about previously when Signal added quantum to their existing Diffie-Hellman, and the triple ratchet. And again, as I said, this was all done. This was validated at the compiled machine code level. I'm not going to get into the details of those four security principles, since even though it's terrific that what was believed to be true and correct of them has now been proven to a mathematical certainty, the real story…

Leo Laporte

Yeah. Very interesting. Very interesting. And do you think ultimately it'll mean better code?

Steve Gibson

It's going to mean better code where it matters.

Leo Laporte

Yeah.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Security Now