François was describing what she told the G7 about open source AI, digital sovereignty and child protection. Her argument is that a government legislating on safety also has to make sure its own builders have the means to comply, or the law ends up favouring the largest companies.
I think this is a call for everybody to roll up their sleeves and work together. We need more attention to this problem. There's a lot of really good civil society groups who are doing these surveys. That's why we know that this is happening and who give us a lot of context on, okay, well, what does that look like for a kid? How does that happen? There are a lot of players who have technology to contribute that can help mitigate this harm at sort of the different areas where it needs to be stopped, right? And we want that technology to be contributed in the open comments. We also do need a philanthropic organization to say, we're going to take a bet on the fact that we can accelerate this work. We can do this in public. We can do this with the rest of the scientific community. And as I said, I think that there are a lot of expertise that live in civil society and in the scientific community that for too long we've said, I know you don't belong in the safety and safeguard conversation. If we want your input, you will sign a stack of NDAs. You will come to some office in Silicon Valley. We will hear you and then you will leave and you will never know what we took from your remarks. That's not a good model. That is not going to give us the pace and scale and excellence that we need in the face of these problems. So I think this is one where everybody can really bring something. I'm somewhat encouraged and also somewhat disturbed by all the conversations that we're having right now on pacing the frontier and how everybody needs to bend together to do something. There is also sub-frontier problems that are very urgent, very catastrophic, and that we absolutely should bend together to focus and solve these problems.
Right. And I mean, and so much of the conversation is also about global cooperation. And what is happening around the world? And so you were recently speaking at the G7 in France, and you were talking about the importance of open source, digital sovereignty, child safety. Can you talk about why this is so important in the context of European builders? And so, like, why is it helpful, especially in that context?
Yes, I was invited to contribute to the G7, and not only because of some of my academic work with a broader coalition of colleagues on defining what we mean when we say open source AI. Because, of course, open source has had a traditional definition that we work through a lot for software. But when it comes to AI, it's a little bit different. And in fact, there are different ways to do open source when it comes to AI. And open source AI can itself take different shapes. It can be just open weights, or it can be everything is open, including documentation and data sets and safeguards, or it could be many different combinations of this is open, but not that. And you sort of have to think about the entire stack. And for each component of the stack, you have to think about what would it mean to open that. And then you think about like, what am I trying to achieve and which shape of openness gets me there? That conversation is very relevant to sovereignty, because, of course, you have the European Union, but also most recently Canada with their big AI strategy saying, well, we want to be able to own, not rent, the different components of the stack that matter to us. That is also extremely relevant for safeguards, right? Because similarly, if you are going to legislate on child protection, you also want to make sure that people, your own ecosystem of innovators, have the means to comply with your law. Otherwise, the more we call for regulation, more rules on this topic without investing on open source tools, the more what you're actually doing is further concentrating power in the industry and closing down the market around a handful of incumbents who have the safety engineering tools and teams that are able to comply with those laws, right? So it's critical to your ability to protect your citizen and to make regulations in this area. And it's also critical to your ability to have independent and strategic and sovereign versions of your stacks.
Well, going a little bit more global, one of the challenges when we deal with a lot of the prospective harms is that, for example, abusive images can be created anywhere. And so, and distribution networks and all the rest. So, what needs to happen globally? Because one of the questions is: there will be image generators, for example, running in the vast majority of countries. There's already open source versions of them or open weights versions of them. And there's nuances you were gesturing at there. So, what needs, what should be thought of as some good basics or some good starts for the G7 coming together, for the UN? We're here in UN General Assembly Week. What would be the kinds of things that would be good things to do globally?
I think my first ask is: never use the idea that you can't root out all of the problem to justify not taking the extraordinarily important incremental steps that really are going to save lives and reduce harm. Yes, there will be people doing this with models that we don't have control of in places that we won't see. But also, when you look at the way the majority of these harms are being perpetrated, it is happening with apps that are being advertised on social media, distributed through app stores, and in context where there's actually so many different levers that we can identify and say, here is a good chunk of the harm that if we focus and join forces, we can remove. So, I'll say that's my sort of first most important role, which is like, let's get progress done. Then we'll worry about the sort of remaining part that we can't get to, but there's so many lives we can save, harm we can remove, good we can do by focusing on all these incremental steps that we're right now not investing enough on.
You've described your generation as internet optimists who believe cyber space would spread democracy
