“It is sometimes incredibly smart. The way I like to describe it is 95% Einstein, 5% Mr. Bean. And if it doesn't tell you when it's the Mr. Bean, then how are you going to automate anything?”
The MAD Podcast with Matt Turck · AI Research & Frontier Labs · October 2026
Almeida is explaining why the reinforcement learning that made language models top the benchmarks hasn't made them good at automation. The models are jagged and unreliable, and they are poorly calibrated about when they are wrong, which is the gap his company says it is working on.
Transcript
…matches the shape of that. And it causes a different type of over promise under deliver. There, RLVR is obviously very, very cool, though. I think, actually, I think RLHF is very cool too. But I just worry about the lack of automation. And empirically, RLVR hasn't really changed automation.
Yeah. So our RLVR doesn't help us in the problem we're discussing because it focuses on rewarding the final solution, whereas what we're trying to do here is probability and understanding. It's also stochastic.
It's also like unreliable. And like I think people like the word jagged. It is sometimes incredibly smart. Like the way I like to describe it is 95% Einstein, 5% Mr. Bean. And if it doesn't tell you when it's the Mr. Bean, then how are you going to automate anything? So it's jaggedness and non-robustness, I think, is the real issue with it because it's very good at answering simple classifications too. But if it doesn't do it reliably, how do you automate anything in the first place? And that's kind of where calibration comes in. And the models are still not very good at that.
Okay, great. All right. So RLHF kind of creates the problem. RLVR doesn't help us. So therefore, you guys had to invent a new post-training method that you call RL CD. Is that correct? And if so, just explain as much as you can share what it is. I
don't know if I would call it a post-training method because when we made RLHF, the concept of post-training did not exist. And you could call RLHF a fine-tuning method, but that undersells how much change you did to the whole process. So no one would just call it fine-tuning, even though maybe…
So to be able to do training, you need to a bunch of data. This is data for. A very specific kind of task in the new paradigm that you're inventing. Where does that data come from?
More from The MAD Podcast with Matt Turck
