Data Skeptic · October 2026
Ju on self-driving cars that stop or hesitate in ways human drivers don't. She says companies tend to blame the people around the car, and that she can't think of another case where a newcomer to an environment isn't expected to follow the local norms.
Oh, okay.
But in both cases, I guess I don't know the state of the machine's memory, but they're showing a data visualization where they recognize where other cars and people are, but those people have no faces, at least on the screen, nor do they have that sort of stick-figure skeleton you see in a lot of computer vision papers. They're just sort of blobs. It implies to me that the car is not at all considering those aspects of the person. Do you think that's a gap they'll have to cross before these can be fully mainstream?
Yeah, I do. I mean, one thing that I know all the autonomous car companies and all the mainstream OEMs that have cars with adaptive driver safety systems, they're all working on this kind of idea of modeling the intent of the other road users. And it's really hard to do properly if you don't, for example, know which direction people are facing. Obviously, as humans, we have a face and eyes, and it's easier to move more and more likely for you to move forward than any other direction. But a lot of the systems right now just recognize a human bomb and even orientation, which can be discerned from the video, cannot discern quickly enough that the car can use it in real time. And so the ability of those cars to model the future intent of people and other road users is really limited by just that limitation. And so there's all these ways that right now, what the car companies are doing is they're playing it safe. So the car will, when it doesn't know what the user is going to do, it's permitted for you to just hang out and go slowly or not go at all. And this makes sense on some level. I definitely don't think the car should lunge into spaces it doesn't understand. But one thing that the automotive industry doesn't want you to know is that these cars get rear-ended at a rate that's like 10 times human drivers. These are not at-fault accidents. All the time, you will hear the statistic that these cars are safer than human drivers because they get into fewer at-fault accidents. And the reason why they have to say the word at-fault is because they get into so many more rear-end accidents. And the reason why that's happening is because the perceptual and modeling capabilities are nothing like what people have. And I think a lot of these car companies are banking on the fact that increases in computation, you know, should make it so that this should go away before long. But it's not great that they're not being 100% honest about the situation. And then that makes you think harder about what it is that we are doing and how is that working? You know, I don't know the number of times I've had people who work at tech companies who should really know better say something along the lines of as long as our cars behave consistently, people will adapt. And that is like a really ugly statement. Sometimes they'll try to frame it as our cars are driving legally and all the other cars should be following suit. But it's overlooking the fact that we don't have to have laws that say that you don't stop suddenly for no reason in the middle of the road because it's not a thing that people commonly do. And when this is a failure mode of some of these cars and they're just trying to cover it up with like not having to report things is shocking. And it's one of the reasons why we need funding that doesn't just come from the auto industry to research this sort of thing. But anyway, so that's an example. Like I think it's actually quite important to understand what it is that people are doing because people expect the cars to do what we're doing. So they'll do things like lean forward with their body. And then when they think the car has seen them because the car is slowed down, they'll start to go. And like there's this really wonderful research by Barry Brown where he's just like looking at footage from people who have been sitting in Waymos and stuff like that, where he documents what he calls the halting problem where the fact that the car slows down and waits to make a decision a lot of times signals to people that they've been seen. So they'll start to go just as the car decides to go. And like it's pretty bad. And in every one of those situations, the car company would say it's people's fault, but it's really the car's fault because it's not behaving normally. And I don't know any other situation where some new actor gets introduced to an environment and isn't expected to follow the local norms. But that's, that's what's happening everywhere. I'm kind of surprised that you haven't seen these things because people will tell me how great these cars are. But when I bring up these things, then people always have a giant collection. Like people who live in San Francisco have a giant collection of things they saw the car do that wasn't normal, you know? So everyone knows it's a problem.
Sure, there's some funny YouTube videos along those lines.
Yeah. Well, I don't think that's okay. That's what I think.
Well, your example just now about that pedestrian leans forward, they see the slowdown, and then they go, even though technically, a heavy motor vehicle still coming at their body. There's like a trust established. I was never explicitly taught that, but somehow I know it. I learned it. Maybe I guess people have the ability to learn that. How do we get machines to learn the same lessons?