Connect your Claude, Codex or OpenClaw for full transcripts on millions of podcasts. Two week free trial, cancel anytime.
Start free trial →Into the Impossible With Brian Keating · Ideas & Essays · November 2026
Keating describes an experiment he calls the Einstein test, to see whether a model can rediscover new physics from raw data. He uses it to explain why he isn't convinced by claims of machine understanding.
…show me that you can reproduce it. And I don't know. Is there anything that AI can do that just straight up reproduce a proof that a genius mathematician or, you know, not prove the Riemann hypothesis, but prove some things that have been landmark proofs in mathematics without human intervention?
…as it is, I mean, it is impressive. I mean, I'm using it more and more also myself. And it's just pretty stunning what it can do. But still, it's not. I mean, to claim that somehow LLMs are the same level with what happens in the human mind, it seems to me a huge and a huge stretch. Yeah.
I'm trying to do what I call the Einstein test. If you take LLM now, if you take Claude or OpenAI or whatever, and you say, tell me about the multiverse or tell me about this and that, it knows so much, not just from the physics, but also from the culture. In other words, there's movies about wormholes like we just talked about. There's movies about even honking and honking radiation, right? So you can't even trust the training data, which is a large fraction of what makes these things unique. So we tried asking the following quote. What if you gave it an LLM lobotomy? What if you took everything about the physical world, just trained it on stuff before 1900? Could it come up with quantum mechanics and relativity? And of course, there are other people working on this and there are people that have actually made these LLMs and actually denatured them, denuded them of any pop culture references after 1930. But that's even too late for quantum mechanics, right? And relativity. And really, we try to do it with just giving it data from the planet Mercury and just this orbital history over thousands of years that JPL has. And it gave us better. Back, you know, it couldn't…
that's a kind of exact, you know, I wrote an article tablet. The
tablet, yeah, I want to talk about the AI thinking and that. Yeah,
that's exactly my point. So I have very, I make various points about the fact that, you know, AI, I could very well imagine that AI can do all sorts of things, but it doesn't change itself, right? Yes. You know, humans are able somehow to design something new. I mean, can it design another version…