“In a lot of non-expert spaces there is a belief that trust and safety isn't going very well online because everybody is trying insufficiently hard. … The darker conclusion is that people have tried pretty hard and this is the best we've got for you. Fundamentally we're bad at this because we're not good at doing it, not because we're secretly good at doing it and holding out on everybody because CEOs love being yelled at by senators.”
Scaling Laws · September 2026
Context
Willner, formerly head of content policy at Facebook and OpenAI, was making the case for treating online safety as an empirical problem rather than a question of corporate will. He and Vaishnavi J had just open-sourced a teen self-harm policy and classifier, and he noted that regulatory pressure can discourage companies from even running the studies, since findings can surface in discovery. Earlier in the episode he described policy writing before fast classifiers as "essentially astrology."
Transcript
…So kudos to them for saying that. Like, why are we not measuring the efficacy of these proposed interventions afterwards? There's no harm in saying we thought this intervention would work. It didn't, or it did. You know, let's look at how to improve this intervention or move to something else.
So this is all evidence as to why I need to create my bumper sticker, which is will support evidence-based policy. So I will send you one when I finally get it out of production. Dave, you wanted to jump in here too?
Well, yeah, this broader point about empiricism, I think goes to a larger, I think that in a lot of sort of non-expert spaces, there is a belief that trust and safety isn't going very well online because everybody is trying insufficiently hard. And I am not here to tell you that all companies are acting in equivalently good faith or trying as hard as they should be. But we have spent like 15 to 20 years and many billions of dollars on this over the last 20 years. And actually the darker conclusion is that people have tried pretty hard and this is the best we've got for you, which is the reason we started centropy is the reason work like the work that we did here is so exciting, which is that like fundamentally we're bad at this because we're not good at doing it, not because we're secretly good at doing it and like holding out on everybody because CEOs love being yelled at by senators. Like that's not the situation. The situation is we're just not very good at doing this because it's very new. It hasn't been studied a ton. A lot of the studied information is locked up inside of companies that do not share with each other and where, frankly, regulatory pressures can create problematic incentives about even doing the…
And so, from Vice's perspective and the work you all are doing to, for example, look through data, identify what is a good outcome, identify what is a bad outcome. You mentioned you had 1,200 labeled examples. 1,200 is good. I imagine 12,000 would be better, and 120,000 might be useful. No, I'm…
It depends. It depends what domain you're talking about. But at least in the sort of automated policy improvement and clarification space, narrowly, the sort of project we, the data to support the project we did here, it's more important that you have a representative set of examples that cover all…
No, and I would just add there that there's a reason we put the work into making this an open source set of policies with the recognition that it is not going to be the right policy for everyone. That actually it's a starting point. It is not where any one company might end up. So in an ideal…
More from Scaling Laws
