“When I showed it to people at Ford, they're like, that is the stupidest thing I've ever seen. And then inside of six months ... those same people at Ford were prototyping autonomous pizza delivery with Domino's pizza using, you know, this coach driver method.”
Data Skeptic · October 2026
Ju describes her "ghost driver" study. A person hidden inside a car seat costume drove a car fitted with fake sensors, so that people on the street believed nobody was at the wheel. The idea came from a prank video.
Transcript
We've run a lot of studies that are not technically deception, but for example, we use a method called Wizard of Oz, where, for example, when we deploy a robot, it's not an autonomous robot that's already been pre-programmed to do everything because we don't know what's going to happen. So if you deploy an autonomous robot among people, usually what you find out right away is all the things you didn't think of that people are going to do that you need to respond to. We use a method called Wizard of Oz because there is a person, the wizard behind the scenes is actually operating the robot. So the people who are interacting with the robot perceive the robot to be autonomous, or at least don't perceive that it's a puppet, but that's really what it is. It's like a puppet with wireless strings. And we use that to understand how the interaction should unfold. We're using not only the natural reaction of the people who are interacting with the robot, but also the intelligence, the sort of social capabilities of the person operating the robot to try to do the negotiation. And that helps us get to the point that we have a cleaner model of how the interaction could unfold or what the problems could be if the robot is designed with some understanding of the social context is deployed in. So there's that kind of deception or suspension of disbelief that is operating in some of our research. It's not a lie, but it's on the other hand not true yet, some of the robot capabilities that we are testing. But I think in order to have people understand what's going on and behave, you know, naturally in the moment, we really need to cut those strings and maintain that suspension of disbelief. I do think, to be honest, that like even if you know that it's a person operating it, it feels so much in the moment like a real interaction with the real system that people would still do the same thing. We've seen lots of versions of that. I all the time, you know, catch myself doing things. And I wrote the grant, I set up the experiment, I hired the students, and I find myself doing things and forgetting, you know, what's really going on. But, you know, there is a way in which the way that things transpire is based on an understanding which is different than reality. So there's that level of what we could call deception.
Well, you've worked on a variety of really interesting projects. I've done some reading on that are good examples of this Wizard of Oz approach. Could you pick one out and share with the listeners so they get an idea what we're talking about?
One example is I was doing research in looking at how people are going to interact with autonomous systems, autonomous cars, in like 2013. And when I started doing that, this is treated as like some sort of fantastical future. You know, cool, it's fun to know what would happen in this very far-fetched scenario, but like when will that be useful? And we did a lot of those experiments looking at driver interaction with autonomous vehicles in driving simulators. So you have a whole chassis car, you know, inside of a thing. Like three screens, 270 degrees around the car. We can set up, you know, all these different driving scenarios in this, what we call like theater for the one person who's the driver, you know, even the human driver to see how they interact with the car when the car takes over driving or when the car hands over control back to the driver. So we would do these kinds of experiments. And the really interesting thing is, like, within like a year of my starting this research, all of a sudden, because Google X started to deploy autonomous cars in Palo Alto, pretty close to where I was at Stanford, things went from feeling like this was kind of far-fetched or at least very far in the future to like, oh, these are relevant problems now. So the simulation is one kind of Wizard of Oz, right? Because we would have something where it looked like the car would driving, but really there was a person behind the screen performing the autonomous driving, you know, procedures of the car. Sometimes we would have that happen programmatically, but sometimes if we wanted to be able to handle all sorts of situations that occurred in response to what the users would do, we would just have a person in there. So one of the things that came up really early on when we started to see these interactions with the actual autonomous vehicles that were being deployed in our neighborhood is that a lot of times the cars, the autonomous cars have been programmed so that when the car was unsure about whether another car or pedestrian would go, it would just hang out and wait for, you know, the other road user to get out of the way before it acted. And one of the researchers at the Center for Design Research at Stanford, where I was, Dirk Rothen Bucker, he was like a bicyclist. And a lot of things that, like, one thing that bicyclists often do is they try to just stay on the bike and not get off so that they can start pedaling right away. And the fact that the cars would do this, I joked about it as a Minnesota standoff, you know, you go, no, you go. The fact that they would do that, it was like present a problem for people who are trying to stay on their bike, you know, and balance. And so he was just like, these things are just so horrible. And we should study how people outside the car interact with the car because those people didn't sign up for this future. Like if you're in the car, either as a safety driver or as a passenger, you elected on some level to join in this like autonomous driving thing, but everyone else on the road basically has to deal with it. And so he talked to other people. We were looking to it. And the problem is like, how do you do this research? And just to cut to the punchline, like later that year, just a couple months later, we saw this hilarious YouTube video. This fellow, Magic of Rahat, had this joke video where he was wearing a car seat costume through drive-ins to scare the people who were at the driving because all of a sudden there was like no person. It was like a ghost, you know. And we were like laughing ourselves silly. And someone said, we should do this before the, you know, that problem they were talking about with bicyclist AV interaction. And then we laughed even harder just imagining it. And then I realized we could totally do that. And so the protocol that we developed was we had a person in a car seat costume, just like Rahat, you know, driving the car in these areas that we have instrumented and did test driving. And just to be just be quite clear, we did a lot to scout it. But then we would interact with people who were on the sidewalk or other vehicles. And there would not be a person behind the wheel. At that time, even in the actual autonomous cars, if you looked to the window, you would see the safety driver. You know, a lot of times the safety drivers would want to do things like wave or gesture or do eye gaze and you shouldn't pay attention to any of those things and they shouldn't signal because the car is making the decision of whether to go or not go. And so the signal that you would get from the person would be wrong. But anyway, when people looked into our car, they would see nothing. And so that's Wizard of Oz, the fact that there's secretly a person driving the car, but for the people who are interacting with the car perceptually, there's like nothing. The car was also in disguise. We had LIDAR and different fake sensors on the car. And then we used that to kind of study how people outside the car would interact with the car. And this worked really well. And one of the really interesting things is how much people didn't pay much attention. You know, we showed some videos to people and they're like, that's some nothing. And look at, I mean, even the fact that people are not freaking out is interesting thing. But one of the things I loved was the reaction we got from the auto industry. We had a lot of studies we were doing with people in the auto industry, Ford, Toyota, Nissan. And I remember particularly when I showed it to people at Ford, they're like, that is the stupidest thing I've ever seen. And then inside of six months, like kid you not, those same people at Ford were prototyping autonomous pizza delivery with Domino's pizza using, you know, this coach driver method. And so in some ways, I think that we did the most to kind of show that this could be done. I know that Virginia Tech, for example, did a really large scale study that took place over many months with many cars. And they were able to kind of take this to scale in a way that with my small team and the budgets we had, we couldn't. But like, I don't think anyone would be doing anything like that if we hadn't invented this, what's called the ghost driver protocol.
It's interesting that in a lot of interactions, people wouldn't react to it. I guess they'd carry on with their usual behavior. Was that universally true, or did some people recognize that there was an absence of a driver and have a different reaction than they would have otherwise in a driverd car?
Yeah, I mean, the reactions spanned the gamut. There were people that noticed it and felt uncomfortable and walked around the car. There were plenty of people who noticed and walked in front of the car anyways. And they would like just put their hand up the way that you would when you think a person's gonna let you go and you just want to, you know, emphasize they should stop. But the thing that was surprising, and when we noticed that people really pushed back, was that a lot of people didn't look at all. And they're like, of course they look. We all know that they look. You know, you make eye contact with the driver. This is accepted knowledge. And we just had all this footage of like people obviously not looking, not reacting because they didn't see anything. And what it looked like they were doing is looking at the front bumper of the car and the front wheel, which is actually the places where it's easiest to see, you know, if the car is moving or not. And they're just really trying to decide if the car is moving. Maybe worse is that people were making the decision to go or not go when the car was quite far away, like 50 feet or so back. And what was going on is that when the driver takes their foot off of the throttle before they even put their foot on the gas, people feel seen. So there's a deceleration that happens because of that foot coming off the throttle. And people are like, oh, it sees me. And then they feel comfortable going. And in some ways, this is the most efficient interaction because persons normally across the pedestrians normally cross the street before the car gets to the intersection. And we actually had to do a lot to prevent that from happening. We had to do a lot of signaling where the car would sort of glide in linearly to the stop, which was at that time what the Google cargo was doing. They were, because they were not actually controlled with the person with the foot and the throttle, they were kind of coming in like very smoothly and mysteriously. And so I think when people don't know what's happening, that is when they look at the driver and try to make eye contact. They're trying to see if the driver see me? Is the person sane? You know, should I go? And if the car interaction is going normally, they don't look. So the thing that we all know, and we all remember that we all do when we interact with drivers at the intersection, it turns out not to be the normal interaction. That is the error recovery. That is the thing that we do when we are less certain, you know, and we're trying to address that uncertainty. And I think when we first said it, people thought we were wrong, clearly mistaken. And then now lots of other people have done the same study, lots of different places. And there are places and situations where people always look, but by and large, what we found, which is that people don't look, is actually now everyone knows this is obviously true, which is super funny.
Every time you go online, you leave behind a little trail of information. What you searched for, what sites you visited, what you clicked. And sometimes you realize just how much is being tracked when you look something up once and suddenly it seems to follow you around the internet. I'd rather not make my browsing history everybody else's business. And that's why I use Proton VPN. ProtonVPN encrypts your internet traffic and helps keep your online activity private. And what I really like is that privacy isn't just a marketing promise for them. Proton has a strict no-logs policy that's independently verified. It's open source, so anyone can inspect how it works. And it's backed by a foundation focused on protecting user rights, not quietly making money from your data. You also get fast connections, built-in ad and tracker blocking, and an extensive server network with compatibility in more than 145 countries. So you can browse, stream, work, or just go down a completely unnecessary three-hour internet rabbit hole, all the while doing it privately. It's easy to get started. Right now, ProtonVPN is offering our listeners 70% off of a two-year plan. When you go to protonvpn.com/slash data skeptic, spelling it out, that's P-R-O-T-O-N-V-P-N.com slash data skeptic for 70% off your two-year plan. That's protonvpn.com/slash data skeptic. Well, I'm frequently a pedestrian, and I have to say, I've never actually encountered an autonomous vehicle, nor one being driven by a person pretending to be autonomous. I've only so far in my life encountered regular drivers, and I find the eye contact to be very important. I almost don't know if I want to cross if I haven't made some sort of connection with that driver. And I didn't really recognize that about myself until reading some of your work and noticing that there's sort of a protocol there in place. Maybe this would be a good transition into implicit interactions. Could you tell us a little bit about the definition of that and how you study them?
More from Data Skeptic
