SH

Sam Herring

Things Sam Says on Podcasts

“They call it trace to environment. And the idea is that you take a bunch of agent traces and synthesize RL environments based on those traces and then train on them. And this could work in a continual learning loop. Every night, every task that you've had your agent do while you're sleeping, it goes and creates RL environments, trains itself on them, and updates.” — Sam Herring, MLOps.community “A lot of that is somewhat unnecessary, and we've only trained models to be like this because we expect that they make a wrong mistake and then correct themselves. And most of these models are good enough. I actually think they could reason less and still get to the right answer.” — Sam Herring, MLOps.community “In SF and New York, they drive around these little three-wheel ticketing cars… I started to do some research and found out they're sold on government auction. In February, the first live demo of Hermes agent that we did on Discord, I got on and showed everyone the end-to-end process of me tasking Hermes to write a script to search government auction sites every day… A listing came up that I bought.” — Sam Herring, MLOps.community

Where to Find Them

Sam Herring has been a guest on MLOps.community .

Recently: “My AI Agent Bought Me A Car” on MLOps.community (November 2026).

What They Said

“They call it trace to environment. And the idea is that you take a bunch of agent traces and synthesize RL environments based on those traces and then train on them. And this could work in a continual learning loop. Every night, every task that you've had your agent do while you're sleeping, it goes and creates RL environments, trains itself on them, and updates.” — Sam Herring, MLOps.community

Asked how agents connect to continual learning, Herring describes an idea from a recent open-weight model paper that his team is building on: turning an agent's daily work and failures into training environments overnight.

MLOps.community · 2026-11-09 Permalink → Listen →
MLOps.community Around 16:09 into the episode
Sam Herring

…really strong talent density in terms of the, you know, there's eight of us and we can train a really good model given there. It's so wild. Time and resources. That's so wild. So we've been sort of biding our time and making sure we can do it right. And so definitely more to come on that soon. How

Speaker 2

does it go to the continual learning? Like explain that now where you're at with all of it. I'm

Sam Herring

glad you asked. I'm glad you asked. So Xiaomi, the latest model is Mimo v2.6 Pro. In that paper, they talk about this brilliant thing that has come out lately and is sort of obvious when you think more about it, but still the idea struck me. They call it trace to environment. And the idea is that you take a bunch of agent traces and synthesize RL environments based on those traces and then train on them. And this could work in a continual learning loop. Like every night, every task that you've had your agent do while you're sleeping, it goes and creates RL environments, trains itself on them and updates. Or you could do, you know, amassing tens of thousands of different traces on data sources that you have. Why not just spin

Speaker 2

up a bunch? Yeah,

Sam Herring

if you have a lot of data of Hermes agent uses, what we do first is what's called failure analysis. So we go through and find the exact points where the agent makes a mistake. Then we cluster those by the mistake type. And once you have a bunch of different piles of, oh, the agent, you know…

Speaker 2

those or is that all all automated pipelines also awesome yes and so again that helps with the recursiveness and yeah huge okay that

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
“A lot of that is somewhat unnecessary, and we've only trained models to be like this because we expect that they make a wrong mistake and then correct themselves. And most of these models are good enough. I actually think they could reason less and still get to the right answer.” — Sam Herring, MLOps.community

Talking about token efficiency, Herring says reasoning models backtrack and second-guess far more than they need to. He calls it his personal hot take, and later adds that his own research found returns from more reasoning flatten out at some point.

MLOps.community · 2026-11-09 Permalink → Listen →
MLOps.community Around 22:02 into the episode
Sam Herring

Exactly, because then it has to read through the documentation. Yeah, so

Speaker 2

that's what I was thinking. Like if you're doing research on token efficiency, like you want that, but you don't have to, it's a

Sam Herring

huge trade-off. And for me, token efficiency comes more in like long reasoning traces. Like I think there's a lot of cases in which models sort of over-reason because they've been sort of trained into this basin where like this longer reasoning answer involves a lot of backtracking and second guessing. I think a lot of that is, I mean, this is like my personal hot take because I think that like a lot of that is somewhat unnecessary and that like we've only trained models to be like this because we expect that they sort of make a wrong mistake and then correct themselves. And like most of these models are good enough. Like I actually think they could reason less and still get to the right, you know, right answer.

Speaker 2

Yeah, we also just kind of assume that the longer it reasons, the more correct the answer will be or the more accuracy we're going to have at the end of the day. And that's true that potentially like a lot of that reasoning is wasted reasoning.

Sam Herring

Yeah, it's just tokens like, and you know, on the contrary, like I did do research on this. I published a research blog about this. It did turn out that the longer you have a model reason, the better it does in a generally constrained setting.

Speaker 2

Intuition is right, but

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
“In SF and New York, they drive around these little three-wheel ticketing cars… I started to do some research and found out they're sold on government auction. In February, the first live demo of Hermes agent that we did on Discord, I got on and showed everyone the end-to-end process of me tasking Hermes to write a script to search government auction sites every day… A listing came up that I bought.” — Sam Herring, MLOps.community

The story behind the episode title. Herring had his Hermes agent watch government auction sites every day until a retired parking-enforcement three-wheeler came up, which he bought and now drives around San Francisco.

MLOps.community · 2026-11-09 Permalink → Listen →
MLOps.community Around 02:56 into the episode
Sam Herring

…with doing it and an hour later I came back to him with a program that, you know, it was like codas on GitHub in an hour. No way. And it could find any clothes currently listed that matched either clothing that you could send it or a picture of something random. Like yeah. So I still use that. A

Speaker 2

random item, just like pottery. Yeah. That is, and it's like, what, what vibes with this? I really am inspired by this pottery. I need that. Yeah. All right. I'm going to go look up that.

Sam Herring

Yeah. Yeah. Yeah. Dude. It's, it's a, um, it was a fun use, and I still use it when I want to find clothes. That's great. Uh, the other thing is in SF and New York, I guess other cities, they drive around these little three-wheel ticketing cars. They're these tiny triangular cars. Yeah. I have seen one in my neighborhood, one or two, and I started to. Some research and found out they're sold on government auction. So, in February, the first live demo of Hermes agent that we did on Discord, I got on and showed everyone the end-to-end process of me tasking Hermes to write a script to search government auction sites every day. And you can find one of these cars in May. A listing came up that I bought. No, and I know you have one. Yes, I tried to get

Demetrios Brinkmann

a bit too far.

Speaker 2

You really, you

Sam Herring

have one, and I drive it around San Francisco all the time.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

Collections They Appear In

AI Builders Hands-On Engineering Podcasts