Things Wendy Says on Podcasts

Yeah, I do. I mean, one thing that I know all the autonomous car companies and all the mainstream OEMs that have cars with adaptive driver safety systems, they're all working on this kind of idea of modeling the intent of the other road users. And it's really hard to do properly if you don't, for example, know which direction people are facing. Obviously, as humans, we have a face and eyes, and it's easier to move more and more likely for you to move forward than any other direction. But a lot of the systems right now just recognize a human bomb and even orientation, which can be discerned from the video, cannot discern quickly enough that the car can use it in real time. And so the ability of those cars to model the future intent of people and other road users is really limited by just that limitation. And so there's all these ways that right now, what the car companies are doing is they're playing it safe. So the car will, when it doesn't know what the user is going to do, it's permitted for you to just hang out and go slowly or not go at all. And this makes sense on some level. I definitely don't think the car should lunge into spaces it doesn't understand. But one thing that the automotive industry doesn't want you to know is that these cars get rear-ended at a rate that's like 10 times human drivers. These are not at-fault accidents. All the time, you will hear the statistic that these cars are safer than human drivers because they get into fewer at-fault accidents. And the reason why they have to say the word at-fault is because they get into so many more rear-end accidents. And the reason why that's happening is because the perceptual and modeling capabilities are nothing like what people have. And I think a lot of these car companies are banking on the fact that increases in computation, you know, should make it so that this should go away before long. But it's not great that they're not being 100% honest about the situation. And then that makes you think harder about what it is that we are doing and how is that working? You know, I don't know the number of times I've had people who work at tech companies who should really know better say something along the lines of as long as our cars behave consistently, people will adapt. And that is like a really ugly statement. Sometimes they'll try to frame it as our cars are driving legally and all the other cars should be following suit. But it's overlooking the fact that we don't have to have laws that say that you don't stop suddenly for no reason in the middle of the road because it's not a thing that people commonly do. And when this is a failure mode of some of these cars and they're just trying to cover it up with like not having to report things is shocking. And it's one of the reasons why we need funding that doesn't just come from the auto industry to research this sort of thing. But anyway, so that's an example. Like I think it's actually quite important to understand what it is that people are doing because people expect the cars to do what we're doing. So they'll do things like lean forward with their body. And then when they think the car has seen them because the car is slowed down, they'll start to go. And like there's this really wonderful research by Barry Brown where he's just like looking at footage from people who have been sitting in Waymos and stuff like that, where he documents what he calls the halting problem where the fact that the car slows down and waits to make a decision a lot of times signals to people that they've been seen. So they'll start to go just as the car decides to go. And like it's pretty bad. And in every one of those situations, the car company would say it's people's fault, but it's really the car's fault because it's not behaving normally. And I don't know any other situation where some new actor gets introduced to an environment and isn't expected to follow the local norms. But that's, that's what's happening everywhere. I'm kind of surprised that you haven't seen these things because people will tell me how great these cars are. But when I bring up these things, then people always have a giant collection. Like people who live in San Francisco have a giant collection of things they saw the car do that wasn't normal, you know? So everyone knows it's a problem.

One example is I was doing research in looking at how people are going to interact with autonomous systems, autonomous cars, in like 2013. And when I started doing that, this is treated as like some sort of fantastical future. You know, cool, it's fun to know what would happen in this very far-fetched scenario, but like when will that be useful? And we did a lot of those experiments looking at driver interaction with autonomous vehicles in driving simulators. So you have a whole chassis car, you know, inside of a thing. Like three screens, 270 degrees around the car. We can set up, you know, all these different driving scenarios in this, what we call like theater for the one person who's the driver, you know, even the human driver to see how they interact with the car when the car takes over driving or when the car hands over control back to the driver. So we would do these kinds of experiments. And the really interesting thing is, like, within like a year of my starting this research, all of a sudden, because Google X started to deploy autonomous cars in Palo Alto, pretty close to where I was at Stanford, things went from feeling like this was kind of far-fetched or at least very far in the future to like, oh, these are relevant problems now. So the simulation is one kind of Wizard of Oz, right? Because we would have something where it looked like the car would driving, but really there was a person behind the screen performing the autonomous driving, you know, procedures of the car. Sometimes we would have that happen programmatically, but sometimes if we wanted to be able to handle all sorts of situations that occurred in response to what the users would do, we would just have a person in there. So one of the things that came up really early on when we started to see these interactions with the actual autonomous vehicles that were being deployed in our neighborhood is that a lot of times the cars, the autonomous cars have been programmed so that when the car was unsure about whether another car or pedestrian would go, it would just hang out and wait for, you know, the other road user to get out of the way before it acted. And one of the researchers at the Center for Design Research at Stanford, where I was, Dirk Rothen Bucker, he was like a bicyclist. And a lot of things that, like, one thing that bicyclists often do is they try to just stay on the bike and not get off so that they can start pedaling right away. And the fact that the cars would do this, I joked about it as a Minnesota standoff, you know, you go, no, you go. The fact that they would do that, it was like present a problem for people who are trying to stay on their bike, you know, and balance. And so he was just like, these things are just so horrible. And we should study how people outside the car interact with the car because those people didn't sign up for this future. Like if you're in the car, either as a safety driver or as a passenger, you elected on some level to join in this like autonomous driving thing, but everyone else on the road basically has to deal with it. And so he talked to other people. We were looking to it. And the problem is like, how do you do this research? And just to cut to the punchline, like later that year, just a couple months later, we saw this hilarious YouTube video. This fellow, Magic of Rahat, had this joke video where he was wearing a car seat costume through drive-ins to scare the people who were at the driving because all of a sudden there was like no person. It was like a ghost, you know. And we were like laughing ourselves silly. And someone said, we should do this before the, you know, that problem they were talking about with bicyclist AV interaction. And then we laughed even harder just imagining it. And then I realized we could totally do that. And so the protocol that we developed was we had a person in a car seat costume, just like Rahat, you know, driving the car in these areas that we have instrumented and did test driving. And just to be just be quite clear, we did a lot to scout it. But then we would interact with people who were on the sidewalk or other vehicles. And there would not be a person behind the wheel. At that time, even in the actual autonomous cars, if you looked to the window, you would see the safety driver. You know, a lot of times the safety drivers would want to do things like wave or gesture or do eye gaze and you shouldn't pay attention to any of those things and they shouldn't signal because the car is making the decision of whether to go or not go. And so the signal that you would get from the person would be wrong. But anyway, when people looked into our car, they would see nothing. And so that's Wizard of Oz, the fact that there's secretly a person driving the car, but for the people who are interacting with the car perceptually, there's like nothing. The car was also in disguise. We had LIDAR and different fake sensors on the car. And then we used that to kind of study how people outside the car would interact with the car. And this worked really well. And one of the really interesting things is how much people didn't pay much attention. You know, we showed some videos to people and they're like, that's some nothing. And look at, I mean, even the fact that people are not freaking out is interesting thing. But one of the things I loved was the reaction we got from the auto industry. We had a lot of studies we were doing with people in the auto industry, Ford, Toyota, Nissan. And I remember particularly when I showed it to people at Ford, they're like, that is the stupidest thing I've ever seen. And then inside of six months, like kid you not, those same people at Ford were prototyping autonomous pizza delivery with Domino's pizza using, you know, this coach driver method. And so in some ways, I think that we did the most to kind of show that this could be done. I know that Virginia Tech, for example, did a really large scale study that took place over many months with many cars. And they were able to kind of take this to scale in a way that with my small team and the budgets we had, we couldn't. But like, I don't think anyone would be doing anything like that if we hadn't invented this, what's called the ghost driver protocol.

Yeah, I do. I mean, one thing that I know all the autonomous car companies and all the mainstream OEMs that have cars with adaptive driver safety systems, they're all working on this kind of idea of modeling the intent of the other road users. And it's really hard to do properly if you don't, for example, know which direction people are facing. Obviously, as humans, we have a face and eyes, and it's easier to move more and more likely for you to move forward than any other direction. But a lot of the systems right now just recognize a human bomb and even orientation, which can be discerned from the video, cannot discern quickly enough that the car can use it in real time. And so the ability of those cars to model the future intent of people and other road users is really limited by just that limitation. And so there's all these ways that right now, what the car companies are doing is they're playing it safe. So the car will, when it doesn't know what the user is going to do, it's permitted for you to just hang out and go slowly or not go at all. And this makes sense on some level. I definitely don't think the car should lunge into spaces it doesn't understand. But one thing that the automotive industry doesn't want you to know is that these cars get rear-ended at a rate that's like 10 times human drivers. These are not at-fault accidents. All the time, you will hear the statistic that these cars are safer than human drivers because they get into fewer at-fault accidents. And the reason why they have to say the word at-fault is because they get into so many more rear-end accidents. And the reason why that's happening is because the perceptual and modeling capabilities are nothing like what people have. And I think a lot of these car companies are banking on the fact that increases in computation, you know, should make it so that this should go away before long. But it's not great that they're not being 100% honest about the situation. And then that makes you think harder about what it is that we are doing and how is that working? You know, I don't know the number of times I've had people who work at tech companies who should really know better say something along the lines of as long as our cars behave consistently, people will adapt. And that is like a really ugly statement. Sometimes they'll try to frame it as our cars are driving legally and all the other cars should be following suit. But it's overlooking the fact that we don't have to have laws that say that you don't stop suddenly for no reason in the middle of the road because it's not a thing that people commonly do. And when this is a failure mode of some of these cars and they're just trying to cover it up with like not having to report things is shocking. And it's one of the reasons why we need funding that doesn't just come from the auto industry to research this sort of thing. But anyway, so that's an example. Like I think it's actually quite important to understand what it is that people are doing because people expect the cars to do what we're doing. So they'll do things like lean forward with their body. And then when they think the car has seen them because the car is slowed down, they'll start to go. And like there's this really wonderful research by Barry Brown where he's just like looking at footage from people who have been sitting in Waymos and stuff like that, where he documents what he calls the halting problem where the fact that the car slows down and waits to make a decision a lot of times signals to people that they've been seen. So they'll start to go just as the car decides to go. And like it's pretty bad. And in every one of those situations, the car company would say it's people's fault, but it's really the car's fault because it's not behaving normally. And I don't know any other situation where some new actor gets introduced to an environment and isn't expected to follow the local norms. But that's, that's what's happening everywhere. I'm kind of surprised that you haven't seen these things because people will tell me how great these cars are. But when I bring up these things, then people always have a giant collection. Like people who live in San Francisco have a giant collection of things they saw the car do that wasn't normal, you know? So everyone knows it's a problem.