The TWIML AI Podcast · AI Builders · November 2026
Sam asked whether the recent stories about companies capping internal token budgets are just about spending less, or about using models more smartly. Sadeepa says people default to the biggest model when they don't see the cost, and compares it to how companies first got control of corporate purchasing cards.
…now we have agents in there. Then you equate that to what is the cost of agents, then you have a true financial equation on saying what is the value you're getting or the benefit you get by introducing AI. I think that's not super mature, but that's where the industry is heading to calculate those.
…read that is more about be smarter about how you're using the technology and like don't choose the biggest model for all of your work if it doesn't really require it. Does that and is that in line with what you're seeing? And like, how do you see folks start to grapple with that technically?
I see that all the time. And like when you have when you have access to all these frontier models, you know, as an end user, if I don't know what the cost to the enterprise, I would just use the best one, right? Or just don't bother even, like, why would I change it? Like, if you have a $20 subscription to open OpenAI, ChatGPT, I'm not going to pick a lower model. I'm just going to use the best one I can, right? So, I think that's what's happening in the enterprise. And not only that, like I know I do this and a lot of folks do that. It's like they experiment, right? We just create stuff because it's like we can. So, I think that's probably one of the reasons that it's just like, it's like I'm thinking like early days of P cards, right? Like, you can just buy anything and then it's you don't have to worry about it. P-cards like purchasing cards. And then the companies put controls in place, like, hey, you cannot buy this, you can have to follow these processes to control the cost. So, equally, I think that that's what's happening in the AI tokens when companies are realizing there's a lot, maybe some waste in there. How do we control that? And if you put the onus onto the individual, at least now we have a control…
And even before we get to hosted externally versus on-prem, there's this big question about routing and like, how do I make it easier for folks to get the right work to the right model? Like, how are you seeing folks tackle that? Yeah,
so there's multiple routing technologies or routing vendors out there today that sits in front of an LLM or as a selector when you have a chatbot or something, you can just pick the router as the endpoint and then router selects where to go. So, and the router's intelligence comes in. Okay, it…
Like, how do you know what to do when? Like a CIO or AI leader at a large enterprise, you see like your token spend forecast going this way, your CFO saying it needs to go that way or at least flatten out a little bit, or like, where's the asymptote? And you have, you know, a handful of technical…