JS

John Schulman

Things John Says on Podcasts

Where to Find Them

John Schulman has been a guest on a16z Podcast .

Recently: “Can AI Learn to Improve Itself? | Recursive Self-Improvement” on a16z Podcast (November 2026).

What They Said

“AI can read all the literature. … But you don't have the experience of being wrong, or of making predictions and then seeing how they fared. So I think there are some ways in which the way we train AI might make them worse at some of these capabilities, but it's a little hard to say.” — John Schulman, a16z Podcast

Asked where his research intuition comes from and when a machine could replicate it, Schulman said much of it comes from personal experience: AI has the knowledge but not the experience of being wrong.

a16z Podcast · 2026-11-17 Permalink → Listen →
a16z Podcast Around 09:13 into the episode
Speaker 4

…just go and carry it out. And then there's taste, which is kind of how you come up with the idea in the first place. The models are still not phenomenally good at taste. They go very deep into things like hyperparameter optimization, essentially like tuning all the little knobs that you can tune.

Speaker 2

Even in that example that you shared, the points where you were kind of nudging, redirecting AI, right, that's still coming from you. And I guess my question is, where does your intuition come from? And at what point do you think a machine could replicate that?

Speaker 1

A lot of intuition just comes from my personal experience doing research. But to get good at a certain research field, you also have to read a lot of the past work in that field and you try to absorb the lessons from all this historical work. A lot of the things that I just described can also be done by AI. Like AI can read all the literature and never mind at all. Yeah, memorization can be a blessing and a curse. Yeah, you don't have the experience of being wrong or of making predictions and then seeing how they fared. So I think there's some ways in which the way we train AI might make them worse at some of these capabilities, but it's a little hard to say.

Speaker 2

What John is getting at is essentially the difference between knowledge and experience.

Speaker 1

Companies will probably also just do RL from human feedback to collect data from researchers on like which proposed ideas are good. And that's kind of like absorbing whatever human intuitions like the human researchers have developed and trying to like bake that into the models.

Speaker 2

Creating automated AI researchers also requires us to have reliable ways to know that their work is producing better models. The measurement itself is important for hill climbing to make tangible improvement. You know, a researcher is by definition, you know, it's more open-ended than solving a…

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

Collections They Appear In