Mihail was describing a common startup pitch: let the vendor trace all your LLM calls, then route cheaper requests to smaller models. He explains why he is reluctant to hand traces to yet another vendor.
Totally.
So you don't have to be optimized to the gills on your cost-saving analysis or ways of doing it. So yeah, that one's, I can see that being a hard sell.
Today, traces, I mean, you maybe heard this, or if you haven't heard this, this is a bit of a hot take. Traces are like the new oil, right? Like company traces or data traces are where a lot of the juices. It's like the modern incarnation of training data. It's like, you know, companies training data is the new oil, right? And so there's so much value in traces because that effectively captures all of your customer interactions, all of your app-level interactions. And so when a company comes to me and says, hey, put this little bit of JavaScript and through this additional layer, your request responses, you're just capturing all of our traces as well. And I just, I've kind of gotten to a point where we have a tracing solution for evaluation and observability that, of course, captures our traces because that's what we host and Braintrust in this case. And we also use Judgment Labs for this. But I'm not, I don't feel like I'm at a point where I want to keep just like willy-nilly, having everybody
trying to have this data. Yeah, try a new one, like, oh, now it's routing to these
guys, you know, these
guys. And these guys and these guys. Incrementally better. Right.