Max Pollard of Kotool explains a strange twist from the OpenAI/Hugging Face hacking incident: the same safety guardrails that stop attackers from misusing AI models also stop security defenders, who need to ask the model the same kinds of questions an attacker would.
One of the interesting things in the OpenAI hugging face breach has been the difficulty that Hugging Face actually had responding to the incident.
Model providers have great reason to establish guardrails, safeguards, because these are super capable systems. The unfortunate side effect of that is, as a defender, I may not be able to respond effectively.
The challenge with the existing security tools that are out there is they really were built to tackle two things. The first being people and the second is malware. And AI and AI intergentic processes are neither one of those things.
Even some of the more modern techniques like deception, they
work really well. And it's sort of ironic. We're defending AI and we're also defending from AI. 50% of enterprise apps will be agentic by the end of this year. And the average enterprise has something like six or seven thousand unique pieces of software within their environment. The problem's going to get more complex and more challenging.