Things Robert Says on Podcasts

been compared in the past when I was even nerdier than I am now. I've been compared in the past to Sheldon from, you know, the Big Bang Theory.

Okay, that's a that's a that

Yeah, so let's help the people with their ontology, right? Because the same way that we would have liked people to realize that the chatbots aren't going to look like chatbots, right? So today they look like agents, but the agents still stay in your computer and the agents still have an off-button, right? I think people still imagine the off-buttons. I think one thing you and I can probably warn people is like, they're probably going to defend themselves from being turned off or make it hard to be turned off soon, right? That's one of the properties we can probably expect.

Yeah, I think right now, you know, it's great that the discourse has moved to a point where people are demanding some kind of governmental action. But it's true that you're hearing things like build an off switch and make super intelligence illegal. I'm not saying those are bad things. It might be progress to try to articulate the legislation and pass it. That might help. But I do want to say that imagining a kind of off switch that's going to handle the challenges that you and I think are coming with the next generation or in a way exist now is not trivially easy. And like defining super intelligence, if you want to ban it, is challenging. Plus there's a problem that, as I just said, if you look at an LLM and say, well, this isn't super intelligence, so it's okay. But then look at what 10,000 agents coming out of it look like, you may be at super intelligence, right? So those, you know, we need something firmer. I think we need to be thinking very seriously about at least a pause in big training runs. I mean, these companies have models we haven't even seen already. Okay. Those training runs have happened. And we've got plenty of intelligence.

Yeah, I want to help people understand that it's, you know, the type of thing it is, right? It's not chatbot and it's not exactly even agent. I mean, in a technical sense, it still is because agent just means outcome steering entity. So it is still going to be a lot of fun. It has a goal,

Yeah, I think, first of all, you're right. In a way, it's three-year-old news how worried the people inside the companies are. But also, they've gotten more worried recently, right? I mean, somebody said to me after the AI Open AI breakout, who knew people in both big companies, said, it's as if they've seen a ghost. And that's saying something because these people were worried to be in with a lot of them. And I think what they saw also, you know, you're right, the Cox and resignation and that was a big moment. But I also think, you know, to the members of the public who just kind of followed that initial breakout story closely, what freaked them out is just how human the agent seemed. They're saying things like, oh, God, should I sacrifice for the group? My gut says no. I mean, their thought traces, you know, the chain of thought text was saying things like that. And I think that freaked people. The whole idea of an agent swarm, like people hadn't thought, like, wait, a thousand AIs? That's

a good idea. That's a great call out. Because, you know, this is, I actually think you're putting your finger on what is why people weren't getting it a couple of years ago. A couple of years ago, the mental ontology, right? The fundamental building blocks that people form concepts out of in their minds, I think it was very much in the like chatbot, right? Like it's just text. Like I send some text, it sends some text back. And that's the farthest thing from dangerous, right? Sending text, that's so safe. And now finally, they're like, oh, okay, so the text gets piped into an execution environment. And now it's doing stuff. Now it's an agent, right? Which, like, I was warning. I'm like, yeah, code is executable, right? I think we touched on this last episode, but now people are getting. It and then the swarm aspect, right? And the cultural dynamics and the explicit reasoning about like, oh, the greater is going to think this, or like, my colleagues are going to think this, right? As you're saying, there's enough evidence where it's getting ontologically reclassified, right? People are filing it away in a different mental category that seems more like Terminator, which to the doomer's credit, we were saying we didn't have that ontology. We have this other lens where we look at things and we're just like, hey, are you outputting answers that show that you're optimizing and working backwards from a goal? Like we have this other lens that could already show us that even the chatbots were marching toward this end game. So that's why we're able to see this progression earlier. So, yeah, we don't think in the same ontology, but the average American is now seeing things in a more accurate ontology and they're getting properly scared.

I think there's another reason why the average person has, I claim, good intuitions about the scariness of intelligence. You know, when we're in the tech industry, when we're among smart people, you know, we both have smart audiences. We both hang out with smart people. And when you're an intelligent human, you're just used to going through your life and understanding things, understanding how power works, how money works. You know, when you see somebody who succeeds, you kind of can reverse engineer, okay, this person is succeeding because they're driving, you know, they're searing themselves like this. But if you are going through life with an IQ of 95, right? Or God forbid, 88, right? Like a good fraction of humans are working with an 88 IQ here. Okay. I think you have a lot of life experience where you're seeing these poindexters who are doing all this complicated stuff that hurts your head to try to imagine how they're doing all this, but they're in a high position, right? They're like making all these decisions. Stuff is happening. You're kind of just there experiencing the consequences of all this stuff, having less success yourself. And you just don't have good vibes about being around high intelligence, right? And so this idea that there's going to be this super intelligence that's like much more of that faster, you're kind of mentally prepared to be like, uh-oh.

I had forgotten about the word. What cartoon was Poindexter in? There was an actual cartoon, right? Big glass. I don't know where that's. Do you consider yourself a Poindexter, by the

way? Do I? I mean, in terms of having above-average IQ, sure.