A host had asked whether better paper-writing from models is just a matter of training. Ittai answers that checking a proof is easy to turn into a training signal, while judging an explanation requires modelling the reader, which is why discovery may outpace human understanding for a while.
…to me is when I use AI as a tool, I will often ask it, like, hey, how can you, how can I improve this piece? Like, what would make this post better? And it provides, you know, a multitude of very precise suggestions about change this, delete that, move this over here. And invariably, they're all
terrible suggestions. I'm not taking any of them. So I wonder how much of this is like a writing thing, and maybe the labs just haven't gotten to perfect that yet. Maybe it is
…this goes back, at least the way I think about this, to my previous point about kind of the short-term prediction, which is, you know, it's kind of surprising that AI can solve the millennial problem and provide a lean proof, but cannot provide a human-understandable explanation. And this goes back to kind of what Dan talked about, human understanding. And to me, the reason is the following. Math is kind of completely universal and has no relation to humans. It's just a lean proof of derivations of mathematics. While human understanding, writing something that is human understandable requires you to have a mental model, a theory of brain of how the listener, what the listener understands, what the reader understands, what are their intuitions, how it learns. And so it is much easier to give a reinforcement task where you have a proof and you have to…
Itai, you've got a great metaphor that you've used to describe this frontier as it's unfolding, where you've talked about it as sort of like a shift into the astrophysics kind of era of math, I believe, is sort of the way you put it, where these things are sort of like telescopes and we have to…
I guess this is my medium-term prediction. And there I think there are basically two paths. So one path is that this discovery frontier and the human understanding frontier will kind of move step by step. We humans will develop these really great abstractions that we can explain all these very…
It'll definitely be a countable gap.