RR Ray Redacted On Smashing Security

“What Microsoft found was that it is possible for those weights to be modified. And we're not talking about a lot of modification. We're talking about, out of thousands, seven or eight numbers changing.”

Smashing Security · Software Engineering & Infra · November 2026

“What Microsoft found was that it is possible for those weights to be modified. And we're not talking about a lot of modification. We're talking about, out of thousands, seven or eight numbers changing.” — Ray Redacted, Smashing Security

Context

Ray is explaining a Microsoft Research paper on backdooring open-weight AI models. Changing a handful of dormant weights can plant a hidden trigger, which gives the episode its "sleeper agent" title.

Transcript

Smashing Security Around 34:45 into the episode
Ray Redacted

But until last Saturday,

Graham Cluley

we knew that people could booby trap or backdoor the

Ray Redacted

code that runs in a model, like the Python libraries or the actual code that runs alongside of it. But no one had ever proven that you could actually backdoor a model by changing the weights. Now, just so you understand, the weights is literally a chart full of numbers. Okay. This is a massive chart that looks like nothing but numbers. So you wouldn't be able to necessarily decode the weights. That's how they're trained. But what Microsoft found was that it is possible for those weights to be modified. And we're not talking about a lot of modification. We're talking about out of thousands, seven or eight numbers changing. And if they target dormant ones in what's called the feed forward layer, that's the layer that basically faces the consumer. That's the last layer in the model, so to speak. If they target a dormant one, they can set up a triggered behavior that was not intended in the model to do something like: here's a perfect example. That's a strike of midnight on New Year's Eve.…

Graham Cluley

I would say it is scary, yes. It's a bit like having a sleeper agent, isn't it? Because you've sort of got this piece of code which you're trusting 99% of the time. And what you don't realize is, because no one really understands these things, you know, that the code you're running on your…

Ray Redacted

it's fascinating. You should say the sleeper thing, because if we think about the old cheesy political spy fiddlers, right? They would have a spy or a person that was trained, a Jason Bourne type, and in the middle of his day-to-day activity, the bad guys might have a trigger word like the…

Graham Cluley

I mean, that is when it gets really scary, though, isn't it? Is when it's Agentic AI, which is going to go wrong rather than the chat bot going wrong and telling you something daft. You don't have to tell me if you're not comfortable doing so, Ray, but do you let an AI actually do things on your…

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Smashing Security