“I had this idea about four years ago to take a large language model, but cut off its knowledge base in 1911. Anything that happened after 1911, just erase it. Anything that happened before, keep it. So lobotomize it. Tell it nothing beyond that point and see, can it rederive what Einstein did four years later? And no, it doesn't seem that it can. It'll keep forcing a solution until it gets an answer, and it can get arbitrarily close numerically, but it can't come up with the insight that actually gravity is not a force. It's a manifestation of the curvature of space and time.”
sputnikintelligence.com
Asked whether AI will one day understand reality better than humans, the cosmologist says language models are good at solving problems they're handed but unconvincing at new insight, and describes a test of whether a model could rediscover general relativity.
Yeah, he's the whole thing. Waking up, he's got an app, he's got courses. He's, you know, it's a huge part of his life. And so, you know, he and I can agree on that, even though we might disagree about the value of religion versus not religion.
…machines actually surpass human beings in understanding reality better than we do? Like you were saying, lots of AI softwares, all these, they're doing research on cancer. And you think that there's a possibility that in a couple of years using AI, you know, cancer can be cured. Yes, sure. I think
they're very good at doing things. These LMs plus GPUs are very good at solving problems that are told to them to solve is valuable. They're less convincing in terms of creating new breakthroughs, innovations on things that we haven't conceived or told them what to do. They're language followers. So you can talk to it. You can program it using English or any language, right? Anything that has a logical structure, even like babies can talk to it. They'll try to figure out what a baby or a dog. I mean, I've seen like crazy things, right? But it's not inventing new languages. My favorite example is like it beat chess in 1993. The last human being that will ever win in chess, you know, played IBM's version of this. It was a machine learning, but very similar to AI. And that was the last time humans will ever be able to win. But people still play chess. My kids play chess, right? You might play it or your listeners might play it. So it's still intriguing. Now, my example is then, yes, no human can ever, you know, beat, but will their computer ever invent an algorithmic, you know, kind of game with structure and rules that's in place for 4,000 years? I doubt it. Right. Or go. Will, you know, it can write like Shakespeare. Sort of, but, you know, can it ever do something as evocative or as like tender and connected to the human soul and spirit? No. I also don't…
Now, for a quick message from Anthropic, the team behind Claude, the AI built for businesses. Imagination doesn't need permission, just room to play. Claude is an AI that works alongside you to give your ideas the space they need to grow, and then it helps you make them real and ready to ship.…
So I'm very worried, you know, but in some sense, I'm excited because it means that physicists will have a place in the future to actually do stuff.
You do talk a lot about falsifiability. And I do want to ask, do you think modern physics is drifting into these ideas that are mathematically sound very interesting and beautiful, but they are very difficult to explain.