SG Steve Gibson Steve Gibson Software developer and security researcher who founded Gibson Research Corporation, maker of the SpinRite disk maintenance and recovery utility and the ShieldsU...

“A proof never establishes that code is correct in some absolute sense. It establishes that the code matches a specification, a precise, formal statement of what the code is supposed to do. And writing that specification is itself a programming problem with its own bugs. Prove the code meets the wrong spec or an incomplete spec, and you have a terrific proof proving the wrong thing.”

Security Now · Software Engineering & Infra · November 2026

“A proof never establishes that code is correct in some absolute sense. It establishes that the code matches a specification, a precise, formal statement of what the code is supposed to do. And writing that specification is itself a programming problem with its own bugs. Prove the code meets the wrong spec or an incomplete spec, and you have a terrific proof proving the wrong thing.” — Steve Gibson, Security Now

Context

Steve on the limits of formal verification. Even with AI writing the proofs, the human job of writing the specification stays hard; in the Signal case, a small team of experts reviewed every spec before the AI proved anything.

Transcript

Security Now Around 2:23:53 into the episode
Steve Gibson

…And then we're going to dig into something. I mean, another, we already talked about AI and this potentially groundbreaking innovation from MIT. Here's another one. The idea that we could actually, finally prove to a mathematical certainty that code is correct and bug-free and does what we want.

Leo Laporte

…about January 8th. Just some of the things coming up. twit.tv slash clubtwit. Help us help you by joining the club. We appreciate everything you do to keep us on the air. Now we continue on, ladies and gentlemen. Join the club, one of us with Security Now. And wow, the case of the provable code.

Steve Gibson

…of needing a functional spec. In 1977, Donald Knuth ended a note to a colleague, and this was famous. I'd seen this a lot. Maybe our listeners will be familiar with it. It's become one of computing's greatest one-liners. He said, Beware of bugs in the code above. I have only proved it correct, not tried it. And so, you know, that was also Knuth's famous wit, but he was making a serious point. A proof never establishes that code is correct in some absolute sense. It establishes that the code matches a specification, a precise, formal statement of what the code is supposed to do. And writing that specification is itself a programming problem with its own bugs. Prove the code meets the wrong spec or an incomplete spec, and you have a terrific proof proving the wrong thing. For something like a sorting routine, the spec is simple. The output is a permutation of the input in order. But what does the formal specification of a secure messaging protocol look like? We need to express mathematically exactly what an attacker can and cannot do. What it means for a key to stay secret. What happens after a device is compromised and later recovers. Cryptographers spend their…

Leo Laporte

Yeah. Very interesting. Very interesting. And do you think ultimately it'll mean better code?

Steve Gibson

It's going to mean better code where it matters.

Leo Laporte

Yeah.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Security Now