“I often say AI wants to be local. Unlike something like Google search, you need massive databases, you need armies of servers … If you look at what Claude and ChatGPT are doing, you can run that entirely on one laptop with no internet connectivity. So the shape of these models is perfect for running locally, unlike almost everything else we've seen over the last couple of decades where the cloud has made more sense.”
sputnikintelligence.com
Asked to look 12 to 24 months ahead for local and open models, Pete Warden gives the thesis behind the episode title. He argues that models, unlike search, don't need a data center behind every query.
Yeah. And I'm always terrible at predictions. So.
But you, one thing Pete is super good at for our listeners is to know what to work on. So that's an indication. Well, just tell us what you're working on.
Yes. So, so what I see from like, you know, from being deep in the technical side, you know, most, most of what I end up working on is because I've been working on something and it becomes blindingly obvious that the only reasonable technical solution is X. And so of course I want to work on X. And what I see and what I've seen for the last few years is the gravitational pull of running this stuff locally or on-prem or however you want to call it. I often say like AI Wants to be local. Like unlike if you think about something like Google search, you need massive databases, you need armies of servers, you need all of this stuff. If you look at what Claude and Chat GPT are doing, you can run that entirely on one laptop with no internet connectivity. So the shape of these models is perfect for running locally, unlike almost everything else we've seen over the last like couple of decades where the cloud has made more sense. So what I expect is all of these pressures, cost for enterprises, especially, worries about privacy, worries about capabilities going away, worries about changes. You know, if you built an application on top of a particular version of Claude, even if you're using the same version in…
the other thing too is I think a lot of enterprises are recognizing that the moat or the remote is really not the model. It's the compounding loop around the model, right? So if you use the model more, you learn more and things like that. So why in the world would you send that to a commercial…
Why not do fine-tuning on your own model? Yeah,
yeah. I mean, own that loop.