“People always say the open weights models are just distilling the proprietary models. But as long as the proprietary models keep improving, the open weights models are going to improve. The analogy I've come to like is water skiing. A water skier is being pulled by a speedboat. The skier can go as fast as the speedboat, or maybe even faster, but can never get ahead of the speedboat.”
sputnikintelligence.com
Listing his bear-case worries from the Anthropic S-1, including falling prices as open-weights models improve, the host offers this analogy for how open models relate to the frontier labs.
…have to incur all of these expensive costs around training, right? Yes. Yes. And by the way, before you continue there, but if you talk to anyone in enterprise, they'll tell you, quite frankly, look, if they don't release another model for the next six to 12 months, we have more than enough to do.
…this type of model switching that even that has been assumed because there is going to be updates, obviously. Every application does it, but it's not going to be at the pace that it is anticipated. So I think that will slow down. And the question then becomes, what's the mode for these companies?
Right, right. And so again, the third item I don't like is that prices are heading down. And you can see this through all sorts of third-party data, right? So the Chinese labs are producing tokens at an industrial scale, releasing their models as open weights. And I think in the next six to 12 months, we'll... Have respectable Western open weights models, right? So NVIDIA is starting to buy Neolabs to beef up Nematron. Reflection has an open weights model. Thinking machines will probably continue to release open waste models. And then I know some initiatives that are going to go do kind of a full open source model initiative, right? So I think the pressure from some notion of open is real. And people always say, well, the open weights are just distilling the proprietary models, right? But then as long as the proprietary models keep improving, the open waste models are going to improve. So the analogy that I've come to like is the water skiing, right? So a water skier is being pulled by a speedboat. So the water skier can go as fast as the speedboat or maybe even faster, but can never go get ahead of the speedboat.
That's right. That's right.
So, you know, prices are heading down. I think that's going to continue, right?
Yeah.