“A lot of that is somewhat unnecessary, and we've only trained models to be like this because we expect that they make a wrong mistake and then correct themselves. And most of these models are good enough. I actually think they could reason less and still get to the right answer.”
MLOps.community · AI Builders · November 2026
Context
Talking about token efficiency, Herring says reasoning models backtrack and second-guess far more than they need to. He calls it his personal hot take, and later adds that his own research found returns from more reasoning flatten out at some point.
Transcript
Exactly, because then it has to read through the documentation. Yeah, so
that's what I was thinking. Like if you're doing research on token efficiency, like you want that, but you don't have to, it's a
huge trade-off. And for me, token efficiency comes more in like long reasoning traces. Like I think there's a lot of cases in which models sort of over-reason because they've been sort of trained into this basin where like this longer reasoning answer involves a lot of backtracking and second guessing. I think a lot of that is, I mean, this is like my personal hot take because I think that like a lot of that is somewhat unnecessary and that like we've only trained models to be like this because we expect that they sort of make a wrong mistake and then correct themselves. And like most of these models are good enough. Like I actually think they could reason less and still get to the right, you know, right answer.
Yeah, we also just kind of assume that the longer it reasons, the more correct the answer will be or the more accuracy we're going to have at the end of the day. And that's true that potentially like a lot of that reasoning is wasted reasoning.
Yeah, it's just tokens like, and you know, on the contrary, like I did do research on this. I published a research blog about this. It did turn out that the longer you have a model reason, the better it does in a generally constrained setting.
Intuition is right, but
More from MLOps.community
