“The safety, in my opinion, comes more from what we call the harness, which is how you hold the model and how you interact with the model than the model itself.”
Sidecar Sync · September 2026
Co-host Mallory Mejias asks whether smaller, cheaper models are also safer than frontier ones because they are less capable. Nagarajan says that is the wrong place to look. He points to a cheap open-weights model that could still do real damage if you spun up thousands of instances with unfettered tool access, and to things like how long an agent is allowed to run as the levers that actually matter.
Transcript
Well, you start off with the idea of keeping the data in an environment that you control. And most associations have challenge with this because the data itself is something they don't really control because it's, you know, it's scattered about in an AMS and an LMS and a SharePoint and all these other places. And they don't have one unified physical location where all their data is under lock and key. So that's, that's a problem that is solvable. And there's a number of technologies that can help you with that. We've mentioned on our pod a number of times the open platform we have called Member Junction, which is totally free to the association community. We knew this problem was going to arise and we built this platform starting five years ago for exactly this purpose to house the business data, both structured and unstructured, that could then basically be used in AI workloads, but in an environment the association had 100% ownership of. So that's the data in the ground as well, in a sense, because it's the execution environment too. There are other ways to do it outside of MJ. This is just the open free solution built for this sector. So we talk about it a lot, but there are other ways to do this as well. There's platforms like Databricks. You can bring data into Microsoft Fabric. There's other places that are environments where you can bring your data in. They all have their pros and cons, obviously.
I want to talk about the idea of controlling the model. So Frontier models are one option, but not necessarily the default. A smaller, faster, cheaper model is often the better fit. And of course, switching should be cheap. Can you help us make the case for the smaller model, Ameth? And I'm also curious, do you think, because we talk about them being smaller, faster, and cheaper, do you also think they're safer because they may not have the overall intelligence level of a frontier model?
The safety, in my opinion, comes more from what we call the harness, which is how you hold the model and how you interact with the model than the model itself. You can do really bad things with small models if you just let them run amok. You could take, you know, something like GPT OSS 120B, which was a model released in August 2025. It's not a very intelligent model. You can run it very cheaply, though. And so if you wanted to spin up hundreds or thousands of instances of this much smaller model, that model could probably do some damage if you gave it unfettered access to a variety of tools and gave it a bad objective. So it's not that the smaller model is inherently safer. It's how you use the models. I would argue that the smaller models, though, are absolutely what you need to be using for most things. You know, I've talked in the past about sending a paper clip from LA to San Francisco or LA to New York and using a jumbo jet to do that, right? It's the wrong vehicle for that workload. And you don't need the biggest, most powerful model to do a small job. It's inefficient. It's slow. It's expensive. And possibly it could be less safe. But most importantly, it's just much more scalable if you use smaller models. The speed you get out of smaller models is stunning too. If you're used to your AI applications just being really slow, probably you're using a big and an older model. A lot of people, by the way, they'll set up some kind of an agent or an app and they'll be running on like, oh, I don't know, GPT-4.0 or something like that. And they just never bother to update it. And you ask them, well, how come you haven't done it? They're like, well, we just haven't gotten around to it or we haven't tested the newer models. But yet, oh, well, it's been two and a half years since that model was considered current. And sometimes those models get deprecated, right? And those models aren't undergoing new safety testing, new safety evaluations. They're just sitting there as they were. So actually being current and leveraging the right models for the job is an important part of your safety posture.
Another thing you can control is the autonomy of your agents. Recursive self-improvement is the thing driving the fear in a lot of the conversations. That we've had thus far. And it's a dial, not a switch that flips when you start using AI. So, Ameth, in a controlled environment, what are the actual levers on how much a system can modify and improve itself?
Well, there's a lot of them, actually. And so, these different knobs or levers that exist on this machine that you're building, you have a lot of control over this. So, one of the things is actually just the budget you give to the agent. So, how long can the agent run for? If you give the agent days and days of time to run, which is, by the way, what happened with OpenAI's example, is they essentially had unlimited run time and they're not monitored really, and they're just doing their thing. Yeah, stuff potentially can happen. There's a lot of other ingredients needed to let things happen, but you can also set budgets that are like, well, you can run 30 times or 50 times or for 30 minutes. So, if you have narrower execution windows, that's both a budget constraint, but it's also potentially a safety component for control. Another thing, though, is really the tools or the capabilities that you give to the agent. So, if you imagine the world's smartest human and you put that person in a room and you give them no access to any tools whatsoever, there's very limited things they could do. They'd probably figure something out over time, maybe, if you gave them unlimited time because they're the world's smartest human. But if you gave them limited time and you didn't give them any tools, there's very limited things they could do. You give certain tools that are more general purpose in nature. They can be very powerful. Like, for example, you give Google search to an agent. That's actually a very powerful tool. If it can search the entire internet, it can get all sorts of information. Well, if you want to give it search, maybe you give it search just to one or two particular domains, or you give it narrow search just to internal document repositories that are scoped even more narrowly, right? So, that's that's part of the idea is that you give it kind of the least necessary level of permissions, which is like a very common thing that people teach in cybersecurity is that you grant the permissions that are needed, not like you don't just say, hey, everyone coming into our HubSpot instance, you're all super admins.
That would
More from Sidecar Sync
