“There's not a lot of data of a camera that's looking down at a glass and deciding how to pick it up. There's not a lot of that. And so that's been the major gap.”
Conversations with Coleman · Ideas & Essays · November 2026
Context
Hughes asked what still holds humanoid robots back. LeBlanc explains that language models had the internet's text to learn from, but there is no comparable body of data about physical movement, so companies are collecting it themselves.
Transcript
Yes.
So how far have your robots gotten in terms of being able to navigate the complex terrain like a human would? What are the limits that they have right now that make them still worse than humans? And do you think all of those limits will be transcended and like over what timeline?
…can all of a sudden get new things out of robots than we've ever gotten before. And, you know, so that's that's what all of these companies are in the are in a competition to do. The difference is with large language models, we have a lot of language and the internet had already compiled as much of human speaking and writing as we could with real world data of what a robot needs, right? There's there's not a lot of data of a camera that's looking down at a glass and deciding how to pick it up. There's not a lot of that. And so that's been the major gap.
Much less data that would show how humans do those things. Yes.
Well, and so that's so, so then you go, okay, if we need some kind of data that's going to make sense to a robot picking up a glass, there are all these different ways to do it. One is to use remote controls on a robot and decide and direct the robot to pick it up and do that over and over again…
So I remember, you know, maybe 10 years ago, the conversation about self-driving cars, especially among podcast philosophy bro types like myself, was, you know, what is the, what is the car going to do in the case of a trolley problem scenario between, you know, you can't go straight anymore…
More from Conversations with Coleman
