Things Emmanuel Ameisen Has Said on Podcasts

“One of the most important things to realize if you use language models or think about them is just this very bizarre fact that to make them is not to understand them. You make them by building this architecture, which is kind of like a scaffold. And then you pass a bunch of data through them and they sort of grow in inscrutable ways.” — Emmanuel Ameisen, Lenny's Podcast: Product | Career | Growth

Ameisen, an interpretability researcher at Anthropic, is explaining his job to Lenny Rachitsky at the start of the interview. The point is why a field devoted to looking inside AI models needs to exist at all: the people who build them don't automatically know how they work.

Lenny's Podcast: Product | Career | Growth · 2026-10-18 Permalink → Listen →

Lenny's Podcast: Product | Career | Growth Around 08:05 into the episode

Emmanuel Ameisen

Thanks for having me, Eleni.

Lenny Rachitsky

So you work on interpretability at Anthropic. Explain what that is. Explain what it is you do.

Emmanuel Ameisen

Yeah. So I work in a team of interpretability researchers. And our goal is to understand how LLMs like Claude work by looking inside them. One of the most important things to realize if you use language models or think about them is just this very bizarre fact that to make them is not to understand them. You know, you make them by sort of like building this architecture, which is kind of like a scaffold. And then you pass a bunch of data through them and they sort of like grow in inscrutable ways. And so that's why it's even a thing that we could be wanting to do to study like how these models work. And so a lot of our research has been on, well, you know, when the models are about to produce one more word, there's actually a whole universe of stuff that happens. And we try to kind of untangle as much of it as we can.

Lenny Rachitsky

There's going to be a lot of mind-expanding, I don't know, elements to this conversation, I expect. One is along these lines, this point that you make that models are not built, they're grown. Speak more to that.

Emmanuel Ameisen

At a really high level, the way a model like Claude works is you kind of train it in these successive phases. Like the first phase, which people refer to as pre-training, is you just take a large corpus of data. And over this corpus of data, you sort of feed the model as input, you know, the, let's…

Speaker 3

This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies all Have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you felt the pain of…

Speaker names from our own diarization · position estimated from where the line sits in the episode

Ameisen is explaining why he thinks human-like language about language models is often the most accurate description. Models train on millions of dialogues, and a chat assistant is itself formatted as a dialogue, so the model is in effect playing the role of a helpful assistant.

Lenny's Newsletter · 2026-10-18 Permalink → Listen →

Lenny's Newsletter

Emmanuel Ameisen

…right? And that in terms of understanding the kind of stuff that happens and predicting the kind of risks that materialize, de-anthropomorphization is actually a useful way to talk about LMs. And I have like a few reasons why I'll get into them, but I think that's probably my main take here.

Lenny Rachitsky

Yeah, please share. Because I know Dorkesh famously, I don't know, fimblessly, put out this whole story of the Huggy Face incident and people criticized them for anthropomorphizing AI. So it's really interesting to hear you think that's actually a useful way to think about it. Talk more about that.

Emmanuel Ameisen

…these sort of like that these models are learning these general ideas, these general concepts. So that's one. So they have like, you know, an understanding kind of of the world. It's not just a surface level, like only the text. You have to understand the process that made the texts. Then there's a second reason, which is that. As part of this training, they also encounter millions of dialogues. Predicting the next character in a dialogue requires behaving like the character in the dialogue would. And so, in that sense, part of their training, I think, can best be thought as training an actor, where it's like, you know, it's reading some play and it has to play all sides because it's predicting all the words. And so, it has to sort of act as this specific character that was scorned in act one and like enacted revenge in act two and is facing the consequences in act three. And to do that well, you sort of like need to model what like the motivations and the feelings of that character might be. And then, another thing that people might not know is when you talk to Claude, when you talk to these…

Lenny Rachitsky

Wow. Oh, my God. There's so much interesting stuff there. How often is your mind just kind of blown thinking about basically we've created this alien being that we are now under trying to understand what we have created? How often, as you watch, you're just like, wow, that's crazy. I didn't expect…

Emmanuel Ameisen

Yeah, a lot. This is just next token prediction on a bunch of text. You know, it's not, it hasn't been like, you know, it goes against so many of our intuitions. We're like, no, the way that you make an intelligence is, you know, you'd have like, you give it lessons, you bring it up, like, you do…

Lenny Rachitsky

Wow. Something that comes up as you say that is what's interesting now is AI is helping us learn and evolve and progress, like AI tutoring and all these things. Just like it's interesting how it now is helping us accelerate our own understanding of the world. And I wonder if like this point you…

Speaker names from our own diarization · position estimated from where the line sits in the episode

Ameisen says that when you hold together how simple the training objective is (predict the next token) and how capable the result is, you can react with denial or with curiosity. He chose curiosity, and this is the example he gives of why denial gets harder to sustain.

Lenny's Newsletter · 2026-10-18 Permalink → Listen →

Lenny's Newsletter

Emmanuel Ameisen

…thing behaves is not like exactly like a human, but man, kind of like something that's mimicking a human is really close and is much closer than a program. Like, a program is a very predictive thing. This is something else entirely. And so, that's why I think it's actually a super useful frame.

Lenny Rachitsky

…Oh, my God. There's so much interesting stuff there. How often is your mind just kind of blown thinking about basically we've created this alien being that we are now under trying to understand what we have created? How often, as you watch, you're just like, wow, that's crazy. I didn't expect that.

Emmanuel Ameisen

…the next word. It's not possible that you could ever match the best specialists in our field. And I think that position just gets harder and harder as we get more empirical evidence that actually that is what's happening. And curiosity is kind of the road I've ended up choosing, right? Which is just like, well, how is this happening? Let's open up. Let's look inside. How could it be that one of the early projects I did at Anthropic is I learned SQL, a programming language, and it took me years to master. And then as a little side project, I got Claude to learn SQL and it was better than me at it. And I'll never be better than Claude at SQL ever again. And it took it, you know, just like this, this kind of like simple process. How could this be? And I think there's so much depth in that question and just so much to explore. And it's to me so fascinating to just every day try to unpack that, try to sort of like peel a few layers of the onions back. Like, okay, well. Clearly, this is happening. The simple objective is giving rise to this super…

Lenny Rachitsky

Emmanuel Ameisen

Hopefully we don't change it just so that we ask, we get students who are like, hey, here's a thousand words. Predict the next one. Just read it all. Keep

Lenny Rachitsky

it all in your head.

Speaker names from our own diarization · position estimated from where the line sits in the episode

Asked whether AI models are conscious, the Anthropic interpretability researcher says it's very hard to say either way. What he does believe is that most people underestimate the complexity that emerges inside the models: shared concepts, planning several tokens ahead, and mixed levels of reasoning.

Lenny's Newsletter · 2026-10-18 Permalink → Listen →

Lenny's Newsletter

Emmanuel Ameisen

…know, when like a mouse gets hungry, like it sort of gets aggressive. What's going on with that? Right. And just like, well, I guess I'll run some experiments to see what the factors are. And so it's a very different kind of study than anything that I think most of us on the team have done before.

Lenny Rachitsky

…on this, they didn't expect to create this new kind of almost life type of being. Do you think about consciousness as a part of this? Just like, do you think AI is conscious? Do you think it'll get to a place where we think it's conscious? Or is that just not even worth our time to think about?

Emmanuel Ameisen

…that they do so by like planning many tokens ahead, that they sort of have this mix between this higher order reasoning and lower reasoning. And all of that just kind of emerges is something that I think is underappreciated by people. And to me, it gives me, let's say, a deeper sense of like, ooh, there's a lot to still understand about how they work. I think maybe that's the main thing is I find that the more you look inside, the less certain you are about how they work. And then oftentimes people that will confidently say like, ah, language models are just X or just Y are the people that actually haven't spent a while looking inside.

Lenny Rachitsky

And there's this, I don't know, thread of history through to how we got here, which is a lot of people did not think we would get to artificial intelligence that is useful by just investing in something that was seemingly very simple and just giving it a lot of compute. And that was famously…

Emmanuel Ameisen

I think it's two things actually, because I think it's not just compute. It's also finding an objective where compute keeps giving you returns. So what does that mean? You know, it's actually not that new of an idea. I mean, you're right. Like compute versus sort of, I would say, expert design…

Lenny Rachitsky

This season's supporting

Speaker names from our own diarization · position estimated from where the line sits in the episode

Things Emmanuel Says on Podcasts

Where to Find Them

What They Said

Collections They Appear In