Tim Scarfe had put the scaling case to him: that needing human supervision is a temporary limitation rather than a permanent one. Becker's answer is that these systems will always require supervision because hallucination is not a defect sitting on top of normal operation — it is normal operation, producing an unwanted result. He says he doesn't love the word for exactly that reason.
…is quite deceptive. And I think I, in terms of how to make sense of it, I think we just have to keep in mind what these systems are at the end of the day. They are for predicting the next word or whatever in the sequence that they've been given. And it turns out you can get pretty far doing that.
…have to kind of, and it's very dangerous keeping going because now we're messing all of our code bases up and we're creating all of this slop everywhere. But just hold on, boys, just wait for a couple of years and the next version will kind of bring it back in check. What do you think about that?
…because they are always going to end up hallucinating. You know, I mean, that's inherent to the way that they work. I say this in the book, but, you know, I don't love the word hallucinate. I know there's been a lot of pushback because it's like, oh, hallucination implies a sort of anthropomorphization of these systems. And I don't love that either, but that's actually not my main problem. My main problem with the word hallucination is it implies that when a hallucination occurs, something different is happening than its normal functioning. And that's not the case. They really only do one thing. And when they're hallucinating, they're doing the same thing that they're doing when they get it right. And so I think that unless we have some sort of major, major breakthrough, and I mean, it would probably have to be a breakthrough that makes, you know, LLMs themselves look like ELISA. You know, short of that kind of really fundamental…
know. And ironically, people aren't really talking much about the hallucination now because they're agentic and they can fix their own stuff. You know, it's more like, you know, if you analogize it like a database query or a program interpreter, it's only as good as the program or the database…
I mean, first of all, he didn't say one or two. He actually said zero to two. He's not sure that we need any. But yeah, I mean, why did he say that? I mean, you should ask him. But he believes in something like a singularity. He believes that if you just throw enough computing power at a machine…
Can we unpack this a little bit? So, you were speaking there a little bit about instrumental convergence, which is this idea that basically its instrumental sub-goals towards whatever it's doing will kind of converge on things like power seeking and bad things that we don't want. So, it's always…
