OpenAI's Karan Singhal explaining why the company built a global network of hundreds of physicians to evaluate model answers instead of leaning on a handful. The question was how a model handles users who want reassurance when the medical picture calls for something else.
…kinds of users and also training the models towards that. And so a lot of our approach has really been like, not only do how do we make the models better for these different users, but how do we bring them along? And part of that is how we do evaluations and communicate about them as well.
…reassurance, but from a medical perspective, they need to hear perhaps something very different. How do you teach an AI to recognize the difference and manage different product objectives there, either from consumer engagement and retention versus sort of correctness and sort of clinical quality?
Yeah, this is super central to how we do both evaluation and training. And so zooming out a little bit and the question of like agreement and disagreement around how we respond and how we evaluate, like one thing you could do and one thing that you could kind of base your approach on here is like we could bring in a few clinicians into, let's say, the OpenAI health team and say, hey, you guys figure out how to navigate these kind of uncertain cases. And I'll ask you about these uncertain cases and you'll help us build evaluations and training data and stuff like this and leave it to those few physicians to figure it out. This is actually not the best approach. So it turns out if you ask five physicians or something this any question, you'll get like eight different answers. And so one of the first things that we did when we first started focusing on health at OpenAI was really build this kind of global physician network of hundreds of experts that we work with kind of across the globe, across dozens of specialties and languages and countries…
it's very important to us that like we take our obligation very seriously when there's hundreds of health, hundreds of millions of health conversations coming in. And so across the evaluation stack and training stock, we have evaluations that track these kind of difficult situations where the user…
There's another example or another instance of disagreement, which is me as the consumer, I ask a medical question of ChatGPT. I then show that response to my doctor during a visit. My doctor loves me, by the way. And then they disagree. They disagree with the response. How should I handle that…
Yeah, I mean, I think you should handle it similarly as if disagreement was happening in some other way as well. Like people do disagree about different things. And I think you have a right as a patient to get, you know, ask for another doctor to take a look and get a sense and go to another doctor…