Kantrowitz had told the story of Meta's Muse assistant putting doctors' phone numbers under the wrong names, which sent his wife across Queens to the wrong specialist. Roy, who works with enterprise AI customers, said this is the hardest thing to explain about agents: they are only as good as the data they can reach.
dear. Which involved me getting yelled at
last weekend. Because, you know, though Muse did put this very helpful list of doctors together, it's very capable standard model. I'm saying this facetiously. Put the wrong phone numbers for the wrong doctor. Like put working phone numbers of some doctors on the wrong, underneath the wrong doctors' names. So my wife booked an appointment from Muse, you know, after making this phone call, the first person that it suggested, thinking that it was going to be the first doctor on the list because the phone number was underneath the list. It turns out she had made an appointment for a completely different doctor that she had called because Muse put it in the wrong place. And so the night before she's supposed to go, she's trying to figure out where she's going. And we thought it was going to be a brief commute into Manhattan, but it turns out that the doctor that she had booked was a multi-train ordeal in deep queens. And she's looking at. me and she's like you're freaking muse man
freaking muse well this is hold on i'm gonna give you my my ai this is probably not 101 but like 301 i think you gotta think about no no no i mean again this is something that like in my own work at writer and with like enterprise customers this is the hardest thing to explain it's if you think about right there why are restaurant reservations so easy it's because they're centralized platforms that have already aggregated and cleaned the data very well i'm guessing most especially direct numbers to doctors we are reliant on maybe google local listings maybe again the zoc docs of the world have done some aggregation they never show up in any kind of ai search for me so maybe they're blocked but like if if it is a data set that is not cleaned and kind of centralized you got to double check man otherwise you're going to end up on the train getting yelled at double check if it's not uh it's why flight again flights and restaurants that's why everyone loves to talk about them because they got the cleanest data sets in the world doctor phone numbers local business phone numbers not even a chance
um so ranjan i did in this conversation suggest why didn't you double check let's just say that was not the right that was
not the right answer
but i'm trying to say what i'm trying to say is um you know i do think a frontier intelligence will probably be more accurate dealing with such type of uh data problems and so that's why you know i i'd almost trust not almost i would trust open ai's dots much more than i would try trust uh metabuse uh for a similar uh activity in the future