Ramez Naam
AI, climate tech, and deeptech investor.
“There is no world in which a software only loop has an exponent even close to one. It's well below that. And that says that these things are diminishing returns. … So, I don't doubt RSI. I doubt the consequences of RSI.”
Razib Khan's Unsupervised Learning · Ideas & Essays · October 2026
Asked why he is skeptical that recursive self-improvement will make AI "go foom," Naam walks through OpenAI's own figures: researcher token use up more than 100-fold in a year, experiments per researcher up only 1.6x. His point is that every input to AI progress shows diminishing returns, so a self-improvement loop can't run away on software alone.
Transcript
Skeptical that recursive self-improvement makes it go foom and hits a singularity. But I'm not at all skeptical that these things are going to keep improving.
Oh, okay. That's actually kind of interesting. Why are you skeptical of recursive self-improvement? Yeah. Where the AI, basically, this just basically means AI designs itself and then all of a sudden it just hits escape velocity.
…then figures out a way to make better algorithms or better hardware, which makes AI better. And then you go through the loop again. You have to have a, there's an exponent, you call it alpha or call it R, depending on whose model you look at, that has to be one or above in that loop. Every time you go through the loop, you have to get more output than input in order for this to be an explosive. There is no world in which a software only loop has an exponent even close to one. It's well below that. And that says that these things are diminishing returns. That's how I view it. I'd say there's one decent paper. There's a few papers like Tom Davidson and Daniel ETH wrote an article, I think, for Epoch AI on proposing a software-only singularity, but they got some important things wrong. I think that's really hard to model the improvement of software as a function of researcher input. They used Stockfish, I think, or one of the chess engines. And…
That was a you've been thinking, you've been thinking about this. I'm just saying, like, wire it down. No, no, no, no, I like I like it, it was great. I think that this, that's why I like ask you these sorts of questions. Um, I guess, like, you know, we're going up to an hour, uh, you know, so I…
Yeah, and I think we're discovering things along the way, like uh, you know, the science of alignment, the science of getting AI to do what we want. Machine learning is just very fuzzy, right? Uh, so there's this stuff we're learning along the way, right? And so, like, I'm not gonna say we know…
Well, I mean, I feel like kind of like, you know, I just feel pretty good after listening to you talk because, you know, you have a good track record in solar. Hopefully you're right on AI. Obviously, it's calling a lot. I mean, it actually aligns with a lot of what I've seen in the workplace. Just…
More from Razib Khan's Unsupervised Learning
