Dwarkesh Podcast · AI Research & Frontier Labs · November 2026
Asked what the actual plan is for keeping superintelligent systems in check if they arrive within a year, Christiano says labs would set the AI systems themselves to work on alignment research, and gives this blunt summary of where that leaves us.
…and like which model checkpoints get released when you have random variability in model quality. You have those dynamics which like make things very noisy over short time scales. And then again, those just get filtered out to like even noisier stuff as we move to the market more broadly.
…they're graded or a million other things. Basically, this is the basic argument about instrumental convergence. There's many different reasons somebody could want power. And so what is the argument? Like, if this happens in a year, how do things turn out okay? Like, what is the actual plan here?
So I think there are a lot of different things someone could hope for. I think they are all at the stage of like, we hope something will happen over that year. We're producing very smart AI systems. They're going to do a bunch of research. It is plausible they will think of something over that year that we have not thought of. That is where we're at right now, mostly. I think if they do not think of something clever that we haven't thought of, it is probably going to end badly. In terms of what are the kinds of hopes you could have. So there is one hope that's just like try and make them good guys. Like they're kind of nice. They share human values. They like flourishing. They're in a position to take over. We have very indirect control over what they want. And then we just take the leap of faith. I think this is like a pretty unability. Appealing option. In general, I…
this is like you're literally having to understand the whole computational graph of superintelligent minds. But
we are writing algorithms which are able to like understand and make deductions about the behavior of some super intelligent mind. It's very, I would describe it as simultaneously a bank shot, which is probably not going to end up relevant to this discussion. Like I think there's, you know, a 10%…
Wait, you're like, okay, here's the thing. It is a 10% chance of working, but it's also the most promising thing.