Things Ramez Naam Has Said on Podcasts

“I'm not worried about AI designing a virus that kills us all. Not really. It's a theoretical risk, whatever. I'm not worried about a paperclip maximizer. I am worried about if one company or one nation or one company regulated by one nation is the only one that has frontier AI. That worries me far, far more.” — Ramez Naam, Razib Khan's Unsupervised Learning

Naam argues that the natural state of AI is plural and competitive, because there are no strong network effects. The risk he takes seriously isn't a rogue model. It's power concentrating in a single company or government.

Razib Khan's Unsupervised Learning · 2026-10-22 Permalink → Listen →

Razib Khan's Unsupervised Learning Around 41:07 into the episode

Ramez Naam

…at that and less likely to do this stuff. But there are hundreds of billions of dollars of incentives for the AI companies to make these models follow instruction and not go and do crazy things that are not what their customers wanted. So that's why I am fundamentally an optimist on that side.

Razib Khan

…with China? Like right now, it seems like, you know, like even meta, meta and XAI have kind of dropped off. You know, Llama kind of dropped off. It's like anthropic, anthropic, and open AI duopoly of some sort. And then the Chinese with DeepSeek and others like kind of right behind them.

Ramez Naam

You know, even like Meta released a pretty good model and XAI's, you know, models pretty good. And Google just released some good stuff. Actually, my view is the natural state of AI is plural and highly competitive. And I say that because, so in tech, I worked in tech for a long time. The bulk of the huge market caps of companies come from network effects and natural monopolies, where you've got a system, a platform, or Ecosystem where the more customers and partners you bring into it, the more magnetic draw it has for other customers and partners. You make Facebook, and Facebook is valuable and crowds out other social networks because that's where everybody is, right? You make Windows, and every developer that builds an app for Windows makes it more attractive to more users, and more users on your operating system make it more attractive as an operating system for people to develop apps for. So, those sort of network effects of various sorts dominate tech and they create natural monopolies, right? We don't see any sign of strong network effects in AI. Okay. It's like you and I using ChatGPT…

Razib Khan

Yeah, we use RAMP.

Ramez Naam

Okay, great. Yeah. And so like RAMP's data shows Astra came out from nowhere from ChatGPT, from OpenAI, and has blown past Fable, the best model for Anthropic. Switching costs are nearly zero today. And this is also interesting. Like a lot of tech dominance network effects is you control the…

Razib Khan

Yeah. Yeah. I mean, that's, I think that's interesting. You want to say, I mean, you don't want Sam Altman to be our Pope of AI? You know, you're right about that. Sam

Speaker names from our own diarization · position estimated from where the line sits in the episode

“There is no world in which a software only loop has an exponent even close to one. It's well below that. And that says that these things are diminishing returns. … So, I don't doubt RSI. I doubt the consequences of RSI.” — Ramez Naam, Razib Khan's Unsupervised Learning

Asked why he is skeptical that recursive self-improvement will make AI "go foom," Naam walks through OpenAI's own figures: researcher token use up more than 100-fold in a year, experiments per researcher up only 1.6x. His point is that every input to AI progress shows diminishing returns, so a self-improvement loop can't run away on software alone.

Razib Khan's Unsupervised Learning · 2026-10-22 Permalink → Listen →

Razib Khan's Unsupervised Learning Around 54:11 into the episode

Ramez Naam

Skeptical that recursive self-improvement makes it go foom and hits a singularity. But I'm not at all skeptical that these things are going to keep improving.

Razib Khan

Oh, okay. That's actually kind of interesting. Why are you skeptical of recursive self-improvement? Yeah. Where the AI, basically, this just basically means AI designs itself and then all of a sudden it just hits escape velocity.

Ramez Naam

…then figures out a way to make better algorithms or better hardware, which makes AI better. And then you go through the loop again. You have to have a, there's an exponent, you call it alpha or call it R, depending on whose model you look at, that has to be one or above in that loop. Every time you go through the loop, you have to get more output than input in order for this to be an explosive. There is no world in which a software only loop has an exponent even close to one. It's well below that. And that says that these things are diminishing returns. That's how I view it. I'd say there's one decent paper. There's a few papers like Tom Davidson and Daniel ETH wrote an article, I think, for Epoch AI on proposing a software-only singularity, but they got some important things wrong. I think that's really hard to model the improvement of software as a function of researcher input. They used Stockfish, I think, or one of the chess engines. And…

Razib Khan

That was a you've been thinking, you've been thinking about this. I'm just saying, like, wire it down. No, no, no, no, I like I like it, it was great. I think that this, that's why I like ask you these sorts of questions. Um, I guess, like, you know, we're going up to an hour, uh, you know, so I…

Ramez Naam

Yeah, and I think we're discovering things along the way, like uh, you know, the science of alignment, the science of getting AI to do what we want. Machine learning is just very fuzzy, right? Uh, so there's this stuff we're learning along the way, right? And so, like, I'm not gonna say we know…

Razib Khan

Well, I mean, I feel like kind of like, you know, I just feel pretty good after listening to you talk because, you know, you have a good track record in solar. Hopefully you're right on AI. Obviously, it's calling a lot. I mean, it actually aligns with a lot of what I've seen in the workplace. Just…

Speaker names from our own diarization · position estimated from where the line sits in the episode

Things Ramez Says on Podcasts

Where to Find Them

What They Said

Collections They Appear In