The host asks whether a future frontier model could simply reason its way to discoveries without a lab. Cubuk answers that even a much better model will still have to run experiments to get results, which is why Periodic Labs is building physical labs that models can use to "tinker with the universe".
RSI. That's the biggest idea of firewall. But does that mean that is it plausible that fable six or something or GPT eight could just have a which has been tuned on these sort of higher order reasoning traces could actually just do this without any logic? Because it's sort of the same thinking process, right?
Yeah, I mean, I think there's decision-making under uncertainty. Obviously, in machine learning, there's noise in running the AI loops. But we do think there's a different set of challenges when you're actually interfacing with the physical world. But also, I think there's another piece too, which is getting the compression of everything into weights is still really valuable. If inference time reasoning was sufficient, all of the Frontier labs would have stopped training at Like GPT-4, and we're like, okay, from now on out, we're going to get really good at inference time improvements. And so we think that by, you know, because of the differences between physical sciences and machine learning, getting that compressed into our own weights will lead to different types of systems and different types of capabilities.
But also, we feel like even if Fable 7 gets really good at, like even better than what it is today, it will still have to run experiments to get results. And the reason for that is machine learning is really good at what's been trained on. But scientific discovery is almost by definition what you haven't been trained on. And that's why we're building these labs so that whether open models or closed models can use these labs to tinker with the universe, because we don't feel like you can make a big discovery without trying things.
Yeah, there's not going to be like, no one's going to zero shot the room temperature actually.
That would be pretty cool. Yeah,
yeah. I mean, as someone who has worked closely with wet labs before, it's certainly I am more skeptical about zero shotting like scientific results than some, but it is something that I think some people might ask. So