Finn was asked what it proved that her team fine-tuned a model for a robot they had never seen, from data sent remotely, without knowing how its actions were represented. Her answer is that the field's working assumption — that each robot platform needs its own model, and that crossing platforms would be very hard — turned out to be wrong. She compares it to how people move between controlling their own body, a car, and a video game character.
But
you guys fine-tuned your model on a robot you'd never seen from data you received remotely without even knowing exactly how its actions were represented. And it worked. So like, what does that actually prove? Because you did a lot of that hard stuff.
Yeah. So one thing that we found that I think is perhaps surprising is that the ability to generalize to different robot platforms and robot embodiments is surprisingly easy. We actually, so this was actually technically not the first time that we had done this. So we had previously, even before starting physical intelligence, we had a project where we're specifically trying to train on train models on multiple embodiments. In the past, people had only trained models for like one robot platform and assumed that like if you're trying to get to work across lots of…
And so just so people understand, this is different universities who have different robots that do not look the same, have sort of different specifications, and they're using your technology and it is still performing better than what they had trained in-house.
Exactly. And the robots, they look different. They might have different numbers of joints. They might be larger or smaller. Their cameras are set up in different ways too. Like where they're mounted, where they're like even completely different orientations of the cameras, the height of the table…
Although it is interesting to watch people who are not familiar with video games try to do video game characters. But yes, it was some experience.