Liron Shapira
Host of Doom Debates, the #1 show for high-stakes debate about AI extinction risk.
“The bottleneck in alignment is having AI whose goal is consistent with what humanity would truly want. But the problem is we haven't written down a Lean proposition that says what humanity truly wants. It's stuck in our head.”
Doom Debates! · AI Research & Frontier Labs · November 2026
Context
Robert Wright asked what a formal approach to alignment would even mean. Liron Shapira relays the objection that proving things is not the hard part of alignment; the hard part is specifying the goal in the first place.
Transcript
…I mean, implementing it is part of the challenge, but it makes me feel a little bit better that like if we can just keep reducing the bugs in the implementation, like every time you find a bug, hopefully you fix it permanently. And hopefully you do converge on this lean that you can really trust.
…like the part of the model where that gets activated, when it thinks this thought, and we want it to think that thought, and so we change it, whatever. But then there's this some idea that you can actually come up with a formal, some formal airtight way of aligning models. What does that even mean?
Right. Yeah. So formal alignment. This, this is a hot topic in the space. There's a lot of smart people who claim they're doing it. Max Tagmark has put out some research papers on this idea that you can formalize alignment. Eliezer Yudkowski puts it really well. He just says, look, I don't personally have an important open problem where all I need is to answer a propositional question in formal logic. Like I don't currently have, I haven't posed a conjecture that if only some prover would prove it true or false, then that would make me progress in alignment. That's not the bottleneck in alignment. The bottleneck in alignment is having AI that what it's trying to do, like the representation of its goal is consistent with what humanity would truly want. But the problem is we haven't written down a lean proposition that says what humanity truly wants. It's stuck in our head.
Right.
Right. So it's all in the writing down. It's not in the proving it. So
if you can't formalize what you want, how can you formalize compliance with it? Right. Is that it?
More from Doom Debates!
