“A neuron is not a fact. The fact is in the connections between neurons. And you can have exponentially more connections between neurons than you have individual neurons. So a set of 10 neurons networked together can represent way more than 10 facts, just because the number of possible paths between them is so high.”

Let's maybe do a little lightning round if such a thing is possible in this context on just like how this work interacts with, or maybe should cause me to update my understanding of some other parts of what I

…of always knew that was somewhat of a general, you know, an oversimplification. How would you update my thinking about facts? Like, do they, do we still think they live in the MLPs? Do they like this sort of one core role per neuron does leave me like, okay, then where are the facts in this thing?

Do we see a different profile of factual recall degradation with sparsity as compared to like arc style tests? They can even do better with sparsity, reasoning they do worse. My gut would say factual would also be like significantly affected by sparsity, but what did you actually observe?

So we did test some MMLU data sets and we're able to recover performance on those pretty well. And that does test factual recall. We do have some tests that we're still working on for like medicine knowledge or chemistry knowledge. But for the most part, we really haven't seen much degradation.…

Yeah, interesting. I guess that really just speaks to like localization, which then brings to mind the nature of experts. The old story about experts was the experts themselves are also not interpretable bad news. You know, it's, we can't really identify what role the experts are playing. It seems…