“We now know that models are able to tell whether they're being evaluated or not. If you are subjecting models to tasks that are transparently some form of test, then they are going to exhibit aligned behavior there because they know that's what they're supposed to do. And then elsewhere, on real-world tasks where it really matters, they might be misaligned and you would never know. It's a big problem. The only actual way to measure alignment is by getting into the flow of people's real work and checking in situ, actually in production.”

Well, it's a very good question in part because we now know that models are able to tell whether they're being evaluated or not. So if you are subjecting models to tasks that are transparently some form of test or evaluation, then they are going to exhibit aligned behavior there because they know that that's what they're supposed to do. And then elsewhere on other types of real world tasks where it really matters, they might be misaligned and you would never know. It's a big problem. And what that means is that the only actual way to measure alignment is by getting into the flow of people's real work and checking whether or not the model is aligned or not, you know, in situ, like actually in production. So that's the perspective that we take on the problem is that by measuring actual production traces from real people in the flow of real work, we're able to see is the model following the guardrails? Because we can check what the guardrails are with an independent auditor and then go and check whether or not the model took the right actions or not, what it touched, what it didn't touch, and report any violations. We can check whether or not the model, anything that it says, whether it's reporting to the user that it changed the file in the correct way and checked every line in the spreadsheet, we can check whether the…