“In a lot of non-expert spaces there is a belief that trust and safety isn't going very well online because everybody is trying insufficiently hard. … The darker conclusion is that people have tried pretty hard and this is the best we've got for you. Fundamentally we're bad at this because we're not good at doing it, not because we're secretly good at doing it and holding out on everybody because CEOs love being yelled at by senators.”
Scaling Laws · September 2026
Willner, formerly head of content policy at Facebook and OpenAI, was making the case for treating online safety as an empirical problem rather than a question of corporate will. He and Vaishnavi J had just open-sourced a teen self-harm policy and classifier, and he noted that regulatory pressure can discourage companies from even running the studies, since findings can surface in discovery. Earlier in the episode he described policy writing before fast classifiers as "essentially astrology."
Transcript
I think, in addition to recognizing that there are challenges with the way we're deploying systems for children and teens right now, and identifying those challenges, asking companies to mitigate those challenges, it would be valuable for policymakers to support approaches like this that are actively building solutions to these challenges. And I'll give you a really simple example that has been, it's been like my bugbear for the last year or so, which is all these different laws and regulations that are coming out speaking about the need for a protocol, a need for an evaluation that, you know, that you do something to mitigate this harm. And so now you have an enormous amount of work taking place in the evaluation space and the protocol building space, but very little backing as to like which one makes sense. How do you measure the efficacy of an evaluation or a protocol? How do you measure the efficacy of the intervention that you're proposing? Great. I mean, yes, I would love to have screen time limits. Yes. What's a good screen time limit? And actually, how do I measure if setting those screen time limits led to a measurable improvement in the mental well-being of children and teens? Like we are not taking that argument to its natural conclusion. And I think there's a little bit of, you know, understandable like reluctance to engage in a space that feels more technical. But there's nothing wrong in working with the people who know how to do this and who can work with you to build out those measurements, build out that analysis, help you with thinking about the honest trade-offs of these interventions. That's what I would like to see more policymakers do. And it feels like we get to the point of recognizing there's a problem, asking folks to address the problem. For example, like you must introduce parental controls. Okay, a company introduced parental controls. Was it effective? Did parents use the controls? Like, how many parents actually have parental controls turned on? I was so impressed by SNAP for being one of the few companies to admit back in 2024 that only 2% of all U.S. teens had parental controls turned on. Like, good for them. They were the only company that would actually say that publicly. Nobody else would even admit their numbers, right? So kudos to them for saying that. Like, why are we not measuring the efficacy of these proposed interventions afterwards? There's no harm in saying we thought this intervention would work. It didn't, or it did. You know, let's look at how to improve this intervention or move to something else.
So this is all evidence as to why I need to create my bumper sticker, which is will support evidence-based policy. So I will send you one when I finally get it out of production. Dave, you wanted to jump in here too?
Well, yeah, this broader point about empiricism, I think goes to a larger, I think that in a lot of sort of non-expert spaces, there is a belief that trust and safety isn't going very well online because everybody is trying insufficiently hard. And I am not here to tell you that all companies are acting in equivalently good faith or trying as hard as they should be. But we have spent like 15 to 20 years and many billions of dollars on this over the last 20 years. And actually the darker conclusion is that people have tried pretty hard and this is the best we've got for you, which is the reason we started centropy is the reason work like the work that we did here is so exciting, which is that like fundamentally we're bad at this because we're not good at doing it, not because we're secretly good at doing it and like holding out on everybody because CEOs love being yelled at by senators. Like that's not the situation. The situation is we're just not very good at doing this because it's very new. It hasn't been studied a ton. A lot of the studied information is locked up inside of companies that do not share with each other and where, frankly, regulatory pressures can create problematic incentives about even doing the studies in the first place, because if you don't do them, they can't turn up in discovery, all the rest of this stuff, right? And so to me, it points in this direction of wanting support. For doing work in public about what better intervention looks like, how we measure it, and how we build better tools. We're organized as entropy as a public benefit corporation because we want to have the flexibility of doing some of the work in public and open sourcing things when it might strictly be in our advantage not to from a commercial point of view. Because we think there just needs to be a much more robust public conversation about all of this that starts with an acceptance that the right answer may not be known yet, and we may need to actually go discover what it is. And that is a process that is inevitably technical and fiddly and involves a lot of testing and a lot of failure, frankly.
And so, from Vice's perspective and the work you all are doing to, for example, look through data, identify what is a good outcome, identify what is a bad outcome. You mentioned you had 1,200 labeled examples. 1,200 is good. I imagine 12,000 would be better, and 120,000 might be useful. No, I'm wrong. That's great. This is wonderful for the audience. I'm getting a bunch of headshakes very clearly that I'm going in the wrong direction. So, tell me why we don't need more data, or tell me why labs or third parties might be able to contribute more information. Basically, I'm asking, how do we get more information that can inform even better classifiers in more domains? Or do we not need that at this stage? Do we have all the information we need?
It depends. It depends what domain you're talking about. But at least in the sort of automated policy improvement and clarification space, narrowly, the sort of project we, the data to support the project we did here, it's more important that you have a representative set of examples that cover all the kinds of problems you anticipate running into than that you have very large numbers of examples. And it's one of the many ways that this approach turns out to be beneficial in terms of making you more nimble, right? There's more nimble because you don't have to have as much human labeling happen. There's more nimble because the classifier can respond to a change in a policy text instead of a relabeling process. And then there's more nimble because you don't need to label as much data in the first place to get this to work. And so it reduces the amount of work in all of those contexts. That's not to say that more data is never useful anywhere in the wide range of things we've covered from like a statistically significant studies of effects point of view. It definitely is. But specifically for this machinery, it turns out to be the case that it's more important that you have coverage of the variety of things you expect to see than that you have the 150th example of that kind of problem. This approach doesn't really need that to work, which is pretty neat.
No, and I would just add there that there's a reason we put the work into making this an open source set of policies with the recognition that it is not going to be the right policy for everyone. That actually it's a starting point. It is not where any one company might end up. So in an ideal world, this is forked like 100, 1,000 times, 100 or 1,000 different companies who all appreciate it as a starting point, but then train it on their own data, their own use patterns, in a proprietary way. We're not asking you to share anything, you know, publicly that you don't want to. And then you can still deliver good outcomes for your users. So I think that is a more valuable outcome from all of this, that and the ability to show your work, that like you have a very reproducible policy that you can repeatedly show achieves the same or mostly the same outcomes based on how your users are engaging on your platforms. So, I mean, I got, I remember someone was asking me this, like, I don't know if I told you this, Dave, but someone was asking me a couple of months ago, like, why would you give this away for free? And I'm like, I'm not, I'm not Mother Teresa. Like, I know that all my clients will still need a forked version of this for their specific needs, and we will work with them on it, but we can make it an easier entry point for a variety of companies that will need this, you know, don't have to like start building this policy from scratch in the first place.
More from Scaling Laws
