On what still stops AI from doing research on its own, Chen separated routine execution from research taste, and said taste is where models lag.
…Mark spectrum, is sometimes referred to as full RSI. The question on everyone's mind, how do we get to full RSI? I spoke to Mark and John about how we might overcome some of the biggest barriers, including teaching models some of the more intangible qualities that exist in researchers, like taste.
Research taste is about knowing which ideas are important in the long run, which ideas really catch on and end up going beyond whatever paper they first appear in, but being general and being composable with other ideas.
There is kind of the boring run-of-the-mill execution. You know, you have an experiment, you already know what the parameters are, and you just go and carry it out. And then there's taste, which is kind of how you come up with the idea in the first place. The models are still not phenomenally good at taste. They go very deep into things like hyperparameter optimization, essentially like tuning all the little knobs that you can tune.
Even in that example that you shared, the points where you were kind of nudging, redirecting AI, right, that's still coming from you. And I guess my question is, where does your intuition come from? And at what point do you think a machine could replicate that?
A lot of intuition just comes from my personal experience doing research. But to get good at a certain research field, you also have to read a lot of the past work in that field and you try to absorb the lessons from all this historical work. A lot of the things that I just described can also be…
What John is getting at is essentially the difference between knowledge and experience.