“An agent is just software, like traditional software. It's running in Python, C, Rust, what have you, and it's calling out to a model. But the kind of workload an agent triggers is lots of tool calling, lots of command line calling, and it's very chatty with the models. Agent workloads are a massively additive workload to whatever you're already doing.”
sputnikintelligence.com
Ryan Donovan asked whether agents change anything about the hardware picture. Spinelli says agents add CPU-heavy load on top of existing demand rather than replacing it.
…typical APIs like Vulkan and other APIs inside your phone. So sorry, again. Complex, you got to go unwind that. But again, we try to do most of the heavy lifting there. And so that and package that in an SDK so that someone can take full advantage of it without having to do all that hard unpacking.
…and I've also heard the TPU, the tensor processing unit, where a lot of these things we're talking about, machine learning graphics, are doing a lot of very complex math, like calculating a lot of numbers all at once. Does any of that change with agents, or is this just sort of like how that works?
Yeah, agents, it's interesting. Agents end up being additive, right? And so it doesn't change an agent, and this is important for folks to realize an agent is just software, like traditional software. So it's running in Python, C, Rust, what have you. And it is calling out to a model, right? And it's probably calling out to a large language model for its deep thinking, and it might be calling to other models for various process categorization or vision or sensor fusion. But suddenly, the kind of workload that an agent triggers is lots of tool calling, right? Lots of command line calling. And it's very chatty with the models. And so agent workloads just are a massively additive workload to whatever you're already doing. And it's increasing the sort of compute demands on your devices. Because
they can move faster and just spit out commands all day, right? You can spit
out commands all day. They can run all night. They don't mind brute forcing things and trial and error. They can definitely run up your token budget.
Everybody's a script kiddie now. Exactly. Part of the efficiency question I've heard is sort of scheduling efficiently on resources. And I think that applies to both the devices and the cloud that you're working on. Are you thinking about software that does that in better ways?