BB Ben Buchanan Ben Buchanan Twitter: @01Core_Ben || Trying to get rich and live forever - happily.

“If I were to pick a blunt instrument, I would probably pick some definition of full RSI or closed-loop RSI, where you're saying this is what we're not going to do. We're not going to just set up a data center and say, go out, make your successor and have no visibility. … A time-bound pause for a pause's sake runs the risk of creating a compute overhang. Then when you do turn the machine back on, you get a big jump in capabilities.”

The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis · AI Research & Frontier Labs · November 2026

“If I were to pick a blunt instrument, I would probably pick some definition of full RSI or closed-loop RSI, where you're saying this is what we're not going to do. We're not going to just set up a data center and say, go out, make your successor and have no visibility. … A time-bound pause for a pause's sake runs the risk of creating a compute overhang. Then when you do turn the machine back on, you get a big jump in capabilities.” — Ben Buchanan, The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

Context

Nathan asked whether they would back blunt limits such as a temporary freeze on frontier training runs. Buchanan prefers banning fully closed-loop recursive self-improvement and explains why a time-limited pause could backfire.

Transcript

The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis Around 1:01:21 into the episode
Ben Buchanan

no, and in fact, we had regulation that they had to tell us. That was executive order, right? Yeah.

Tantum Collins

…runs that are happening at the trillion dollar labs. And that makes it, honestly, pretty difficult, as for all intents and purposes, a member of the general public, you know, an outside commentator on these things to say which Blunt instruments make sense and when should they be put into effect.

Ben Buchanan

…the blunt instrument I would go for. I do think on this question of a temporary pause or whatnot, what I would want to know the answer to is: are we creating a huge compute overhang? What are we doing with the time? And I could imagine circumstances where we say we want to not proceed to some level of capability without meeting safety and security standards. In fact, I probably support that. A time-bound pause for a pause's sake runs the risk of creating a compute overhang. Then when you do turn the machine back on, you get a big jump in capabilities. And I'm not sure that that is a good thing. So if you ever intend to turn the machine back on, if it's a pause as opposed to a ban, then I think I want the pause linked to safety thresholds and security thresholds rather than to, which is what we tend to call pacing, rather than to some time period. But look, I would, you know, do cartwheels if this was a kind of nuanced conversation that the…

Nathan Labenz

Yeah. Well, even at the level of like Anthropic, the previous RSP sounded a lot more like that. And the new one is a lot less like that. So, you know, even among the kind of thought leaders in that space, we've drifted, I would say, the other direction. Do you have legislators that you're excited…

Ben Buchanan

Yeah, Scott Weiner, Manny Rutenell in Colorado is great. So I think there's more, there's more, I would look at it in two ways. There's more folks who, you know, came closer to running on AI safety or being deep on AI safety than ever before. We did not have a Scott Wiener type or a Manny Rutnell…

Nathan Labenz

Yeah, the wave of ballot initiatives on data centers has been quite

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis