Dwarkesh Patel
Hosts the Dwarkesh Podcast, a long-form interview show on AI progress, timelines, and the people building frontier models, along with economists and historians.
“When I had Dario on the podcast, the thing I asked him was: if you truly expect models which will be human-like in their ability to learn on the job, why would you try to bake in all these skills of working with PowerPoint or something? Why not just expect the model to be able to pick that up while it's deployed?”
Dwarkesh Podcast · AI Research & Frontier Labs · September 2026
Context
Pressing the panel on why labs still invest so heavily in domain-specific RL training when their own stated bet is on models that learn continually. Schulman's answer is that if models were good enough at in-context learning you wouldn't need to train them on finance — which is the point, and also the concession.
Transcript
…second is it is that enough like basically learning how to learn within the simulacra within a data center uh and then but getting deployed into the real world But not actually learning from real-world deployment, only learning these meta-skills from the simulated environments in the data center.
…just so much Excel data and all that sort of stuff in the RL training. And then it's PowerPoints. It's this long tail of the working economy. And that seemed to work really well. And a lot of the other labs, I think, even the open source labs have now realized that that was the correct answer. But
what is the implication from that? When I had Dario on the podcast, the thing I asked him was: if you truly expect models which will be human-like in their ability to learn on the job, why would you try to bake in all these skills of working with PowerPoint or something? What did you just expect the model to be able to pick that up on while it's deployed? And so, yeah, there's multiple different explanations. One is just that this is, we expect models to get there soon, but they're not there yet. So why not amortize these skills into the model training? Another is that we're not concentrated on making it really good at widely deployed work. We just want it really good at RSI. And this is just a way for us to get revenue so that we can pour it back into a model that is actually really good at doing RSI development. And then once the singularity happens, the thing that comes out the other end will be really good at all the things which seem like bottlenecks to the current generation of models. Yeah, John, I don't know if you have takes on how once you construe why there is so much task-specific…
Yeah. I mean, if the models were good enough at learning in context, then in theory, you wouldn't need to train them on finance. They would just be able to figure out, read all the books on the fly and figure out how to do everything in the appropriate jurisdiction. Yeah, and you could argue that…
I think another thing is just that it's not that expensive to do both at the same time, right? Because the models are massive. They can easily afford in terms of their parameters to learn everything. And there is likely some transfer and sort of even just even if finance is not specifically like…
That makes sense. Oh,
More from Dwarkesh Podcast
