“We see the code it generated exactly. It's like an alien code, spaghetti. But for some reason, it generalizes really well… It generalized better than our, we would argue, more elegant, hand-tuned harness.”
Machine Learning Street Talk · AI Research & Frontier Labs · September 2026
Context
Jiang's team left a research agent running to improve its own harness, then tested the result on held-out benchmarks including weather forecasting — far outside anything it had been optimized for. The code it produced was unreadable and beat the harness his team had designed by hand. He admits they are torn between sticking with the clean, first-principles version and living with the one that works better.
Transcript
…of efficiency can overcome the diminishing return. So that, yeah, the curve of effort output turn from concave to convex. Of course, that's like the grand goal, like AGI. No one knows how far we can go, but improving the efficiency of self-referential research is definitely already on the roadmap.
…towards improving the agent itself. And what that means is improving the harness. So, you know, the prompts and the tooling and, you know, all of the skill surface, all of the things that the harness can do. And they just left it running. They just left it running and they saw what happened.
There's an interesting thing we observe on self-evolved auto-research harness. We see the code it generated exactly. It's like an alien code, spaghetti. But for some reason, it generalizes really well. Of course, we have a set of benchmarks we're trying to hear Kaiber on. After that, we will test it on holdout benchmarks. Those are public benchmarks, like MLE Bench Light, ARE Bench Light. ARE Bench is like algorithmic discovery benchmark from Sarkana. MLE Bench is the OBI's machine learning engineering benchmark. And we also test it on a very far out of distribution task called Weather Bench…
Do we even need those constraints at all?
That's a really good question. I think in practice, you still need those constraints. Human and AI need to collaborate. There are certain aspects of human intelligence are still superior than AI. I believe so. For example, in Aiden, where we send the agent to attend OpenAI's hiring challenge, we…
So what do we mean by recursive self-improvement, right? Are we talking about building a machine God? Are we just talking about some code that runs in a loop and optimizes itself? We try to pin this down.
More from Machine Learning Street Talk
