“The question is not: does Claude obey you? Does ChatGPT obey you? The question is, does it share your goals and do you share its goals? And the strongest evidence that you have about whether or not something is aligned with you is whether or not it acts like you're aligned with it.”
sputnikintelligence.com
Former interim OpenAI CEO Emmett Shear, now running the alignment company Softmax, argues that obedience is the wrong test for AI alignment and that shared goals are the right one.
…data. That's the training. Do you want to train alignment? Well, then you need, that's the better lesson. You must have training data where people attempt alignment and you get differential, variational outcomes where sometimes you succeed and sometimes you fail. And you need the whole thing.
Right. Right. I mean, it's a tall task, but clearly an important one and one that I think people are still trying to. I mean, when it comes to the safety and long-term success of these agents and tools, that is the one that needs to be addressed, right? So
the question is not: does Claude obey you? Does ChatGPT obey you? The question is, does it share your goals and do you share its goals? And the strongest evidence that you have about whether or not something is aligned with you is whether or not it acts like you're aligned with it.
Right.
And the AIs, the AI alignment that we're looking for, like humanity needs, is that you, you probably chat with, what's your favorite model? What do you chat with right now? What do you think? Honestly, instinct. Instinct, great. I'm
interacting more with that than with.