“The more we call for regulation, more rules on this topic without investing on open source tools, the more what you're actually doing is further concentrating power in the industry and closing down the market around a handful of incumbents who have the safety engineering tools and teams that are able to comply with those laws.”

I think this is a call for everybody to roll up their sleeves and work together. We need more attention to this problem. There's a lot of really good civil society groups who are doing these surveys. That's why we know that this is happening and who give us a lot of context on, okay, well, what does that look like for a kid? How does that happen? There are a lot of players who have technology to contribute that can help mitigate this harm at sort of the different areas where it needs to be stopped, right? And we want that technology to be contributed in the open comments. We also do need a philanthropic organization to say, we're going to take a bet on the fact that we can accelerate this work. We can do this in public. We can do this with the rest of the scientific community. And as I said, I think that there are a lot of expertise that live in civil society and in the scientific community that for too long we've said, I know you don't belong in the safety and safeguard conversation. If we want your input, you will sign a stack of NDAs. You will come to some office in Silicon Valley. We will hear you and then you will leave and you will never know what we took from your remarks. That's not a good model. That is not going to give us the pace and scale and excellence that we need in the face of these problems. So I think this is one where everybody can really bring something. I'm somewhat encouraged and also somewhat disturbed by all the conversations that we're having right now on pacing the frontier and how everybody needs to bend together to do something. There is also sub-frontier problems that are very urgent, very catastrophic, and that we absolutely should bend together to focus and solve these problems.

Right. And I mean, and so much of the conversation is also about global cooperation. And what is happening around the world? And so you were recently speaking at the G7 in France, and you were talking about the importance of open source, digital sovereignty, child safety. Can you talk about why this is so important in the context of European builders? And so, like, why is it helpful, especially in that context?

Well, going a little bit more global, one of the challenges when we deal with a lot of the prospective harms is that, for example, abusive images can be created anywhere. And so, and distribution networks and all the rest. So, what needs to happen globally? Because one of the questions is: there will be image generators, for example, running in the vast majority of countries. There's already open source versions of them or open weights versions of them. And there's nuances you were gesturing at there. So, what needs, what should be thought of as some good basics or some good starts for the G7 coming together, for the UN? We're here in UN General Assembly Week. What would be the kinds of things that would be good things to do globally?

I think my first ask is: never use the idea that you can't root out all of the problem to justify not taking the extraordinarily important incremental steps that really are going to save lives and reduce harm. Yes, there will be people doing this with models that we don't have control of in places that we won't see. But also, when you look at the way the majority of these harms are being perpetrated, it is happening with apps that are being advertised on social media, distributed through app stores, and in context where there's actually so many different levers that we can identify and say, here is a good chunk of the harm that if we focus and join forces, we can remove. So, I'll say that's my sort of first most important role, which is like, let's get progress done. Then we'll worry about the sort of remaining part that we can't get to, but there's so many lives we can save, harm we can remove, good we can do by focusing on all these incremental steps that we're right now not investing enough on.

You've described your generation as internet optimists who believe cyber space would spread democracy