SG Steve Gibson Steve Gibson Software developer and security researcher who founded Gibson Research Corporation, maker of the SpinRite disk maintenance and recovery utility and the ShieldsU...

“The most important takeaway here is that at no point in any of this is there any attempt to understand the value of the many hundreds of billions of individual weights inside the model. We just stand back and the magic somehow happens. LLM technology is far beyond human understanding at the level of its weights. By their own frequent admissions, even those who do this do not fully understand how their own creations work.”

Security Now · Software Engineering & Infra · November 2026

“The most important takeaway here is that at no point in any of this is there any attempt to understand the value of the many hundreds of billions of individual weights inside the model. We just stand back and the magic somehow happens. LLM technology is far beyond human understanding at the level of its weights. By their own frequent admissions, even those who do this do not fully understand how their own creations work.” — Steve Gibson, Security Now

Context

Steve was explaining a new MIT paper in which one AI model reads and deliberately edits the weights of another. He steps outside his usual security beat to say why he finds it foundation-shaking.

Transcript

Security Now Around 2:21:02 into the episode
Steve Gibson

Okay.

Leo Laporte

All right. On we go.

Steve Gibson

So, what is weight space programming? We all know that this podcast is not directly about AI. Twit has intelligent machines podcasts for that. But AI is obviously rocking the boat of everything we do on this podcast regarding cybersecurity. And so for that reason, and just for the sake of my own edification, I previously invested a bunch of time working to understand AI as much as I could and sharing what I've learned about it along the way. So when something truly fundamental and significant happens in AI, since we now have a foundation for putting such events in context and understanding them, I'm going to take a moment to keep everyone current. Such a thing recently came to light with the publication of research out of MIT's AI lab, which has made a rather amazing AI-enabled breakthrough in understanding and deliberately modifying the hundreds of millions of weights within full-size neural networks. As we've covered before, pre-training is the process of training a model to predict next tokens by using gradient descent to tweak the values throughout a huge network to improve its prediction. That process ingests a massive volume of language and then after training, well, I'm sorry, ingests a massive…

Leo Laporte

Wow. What a show this has been. We'll get to that in just a bit. But first, let me make a plea to all of our fabulous listeners, those of you listening right now, to join Club Twit. We're coming to the holiday season, and this is a great time to do it. Club Twit makes a huge difference in our…

Steve Gibson

So, yes. Last Tuesday, a week ago, November 10th, Google DeepMind published a research post titled Introducing Alpha Proof Code, a breakthrough in verifying complex software at depth. Okay, so a little bit of background. And this is, I warn you, going to kind of be a conceptual thinking piece here.…

Leo Laporte

Yeah. Very interesting. Very interesting. And do you think ultimately it'll mean better code?

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Security Now