“When you first learn to drive, you're taught that instead of looking right in front of your wheel, you'll have a much more stable ride if you look out at the horizon. I think there's a similar situation here. If you'd said five years ago that we'd have AIs proving math conjectures, making art, and earning hundreds of billions of dollars in wages, but also egregiously cheating in ways that break laws and commit felonies, it would have sounded so wild. But the general shape of things was something you could have started reasoning about even then.”

I mean, when you first learn to drive, you were taught that instead of looking right in front of your wheel, you'll have a much more stable ride if you look out at the horizon. I think there's a similar situation here. I think you're right, where if you did say five years ago that we will have AIs that are proving math conjectures and making art and contributing tens and soon to be hundreds of billions of dollars of earning tens of or hundreds of billions of dollars of wages, but also egregiously cheating in ways that break laws and committing felonies. It would just be so wild. And you might have been inclined at the time to talk more about extremely practical, direct consequences of GPT-2 or something. But these are in some sense, you obviously couldn't have foreseen a lot of the specific details, but the general shape of things you could have started to reason about even then. But it would have been hard to do so. And so I do feel quite confused. But I do feel like the important thing, one thing I've been thinking about with the podcast is the important thing is to have the conversation I wish I had the way you would have hoped you would have been talking about AIs like the present ones in 2016, rather than talking about rando bullshit about, I don't know what the topic of conversation was in 2016. I think in maybe 10 years…