Solomon, an MIT professor, was asked whether AI models flatter people about mathematics the way they do about everything else. Judging by his inbox, he says, yes. He tried a few open problems from his own field on the tools: they didn't solve them, proved something he found superficial, and then offered to format it for a journal.
…support teams, thousands of experts answering your call at 2 a.m. like it's 2 p.m. Always on, always ready. That's a lot of coffee. One partner powering how business gets done for companies of every size all around the globe. When you add it all up, no one does business like Comcast business.
…themselves, does sycophancy exist when it comes to math prompts the same way it exists for other, I guess, language-based queries. Does the model go like, oh, what an interesting problem is? You're on the right track. What's an interesting problem? You're on the right track. Like, maybe, you know,
judging from my email box, I think the answer is yes. So I'll tell you a funny anecdote. Right. So all the time I get emails from people that are looking for an expert to maybe verify, you know. Oh, yeah. And what I've noticed lately is I get a lot of emails from outside folks that sort of they've been playing with their AI system. They've proven a new result and the AI told them it was really important and they need an MIT professor to validate, which is absolutely fascinating. In fact, for fun, I played with some of these tools and put in a few open problems in my own domain. Interestingly, they did not get solved. So apparently my corner of math is safe at the moment. But the first thing that it did, it proved some, from my perspective, pretty superficial thing. And then it had a little footnote on the bottom being like, as far as I can tell, this doesn't appear in…
So you can imagine a future where I guess you're sort of overwhelmed with new proofs and the bottleneck becomes the ability to actually verify them with humans.
That is a problem right now. And in fact, it's not just in mathematics, it is across academia. So to give you an example, actually, interestingly, I would say the academic field that's been affected most by this phenomenon you're talking about is actually AI machine learning itself.
Oh, yeah.