“One of the most important things to realize if you use language models or think about them is just this very bizarre fact that to make them is not to understand them. You make them by building this architecture, which is kind of like a scaffold. And then you pass a bunch of data through them and they sort of grow in inscrutable ways.”
Lenny's Podcast: Product | Career | Growth · In-Depth Interviews · October 2026
Ameisen, an interpretability researcher at Anthropic, is explaining his job to Lenny Rachitsky at the start of the interview. The point is why a field devoted to looking inside AI models needs to exist at all: the people who build them don't automatically know how they work.
Transcript
Thanks for having me, Eleni.
So you work on interpretability at Anthropic. Explain what that is. Explain what it is you do.
Yeah. So I work in a team of interpretability researchers. And our goal is to understand how LLMs like Claude work by looking inside them. One of the most important things to realize if you use language models or think about them is just this very bizarre fact that to make them is not to understand them. You know, you make them by sort of like building this architecture, which is kind of like a scaffold. And then you pass a bunch of data through them and they sort of like grow in inscrutable ways. And so that's why it's even a thing that we could be wanting to do to study like how these models work. And so a lot of our research has been on, well, you know, when the models are about to produce one more word, there's actually a whole universe of stuff that happens. And we try to kind of untangle as much of it as we can.
There's going to be a lot of mind-expanding, I don't know, elements to this conversation, I expect. One is along these lines, this point that you make that models are not built, they're grown. Speak more to that.
At a really high level, the way a model like Claude works is you kind of train it in these successive phases. Like the first phase, which people refer to as pre-training, is you just take a large corpus of data. And over this corpus of data, you sort of feed the model as input, you know, the, let's…
This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies all Have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you felt the pain of…
More from Lenny's Podcast: Product | Career | Growth
