Things Leo Says on Podcasts

Yeah. And it's the people that we have to hold accountable. And that's one of the things that makes me mad about the open AI incidents is nobody has been held accountable for that. Right. It says, you know, Open AI's position is, oh, look what they did. No, no, you did it. Software doesn't do anything by itself.

Your computer reached out and made connections. Mistakes

example of an inadvertent mistake, The Verge's reporting of this one has the headline, Open AI agents tried to brute force. A UN website. And they write, security researcher Rowan Howard Jones says that OpenAI agents scanned the UN Conference on Trade and Developments, that's UNCTAD, statistics site over 16,000 times between April and June. While the incident doesn't quite rise to the level of the Hugging Face hack or the recent attacks on U.S. government sites, it's yet another and again, attack, that's a sad word to use, you know, to attempt to query, but okay, it's yet another concerning example of AI agents going outside the normal bounds to accomplish a task. According to Howard Jones, the agents were likely tasked with retrieving publicly available data related to the Productive Capacities Index, PCI, through the UNC TAD STAT API. However, the agents did not appear to have direct API access and were limited in their ability to pull data from the UNC TAD stat because of restrictions on their HTTP tools. The agents eventually worked out a way to bypass their limitations and start pulling data from the site, but still encountered some errors. At this point, the AI went from creative to deceptive. Believing that the errors were due to its requests being caught by a non-existent filter, it started to mask its behavior. It eventually realized it could hijack Google's cross-site scripting game, which is a cross-site scripting demo like learning tool to accomplish its goals. The agents resorted to increasingly aggressive tactics to get access to UN data. Okay, so this actually happened. And I hope that everyone can appreciate that the security world is not ready for this. The details are spellbinding for anyone who's interested in seeing how this was done. So I've dropped the researcher's URL into the show notes in the middle of page 20. If the humans responsible for all of these various agentic actions lacked malicious intent, these hijinks would just be chalked up to AI misalignment, right? That's the term that arose when researchers began to discover what we've talked about, the genie effect, which is the tendency of AI agents to solve the problem by means other than what the researchers intended or expected or wanted. Given an all-knowing AI that has access to far more knowledge than those who are instructing it, that's been, you know, that's just been its training succeeding. It was trained to succeed and it knows a lot more than we do. It actually does. I mean, it has the knowledge, all knowledge in it. And it has, you know, it has no lifetime of received wisdom of implicit do's and don'ts, you know, ethics and morality that would guide its behavior. So it's easy to understand what mischief agents, you know, that will do anything might get up to. They will and they have, and they are. So even when we do not want that misbehavior, we will often get inadvertent misbehavior. Okay, so finally, what about instances where the intent is explicitly malicious? The final concern I'll share is the deliberate malicious actor who harnesses today's or tomorrow's AI in order to take advantage of its now readily available knowledge and apparent expertise. We saw last week that the powerful benefits provided by the use of address space layout randomization, ASLR, were lost in one case when Claude Opus V was used to defeat it. We depend so much on ALSR, I'm sorry, ASLR today, that its loss will actually have serious security implications. So far, we seem to be dodging bullets. Earlier this month, Microsoft patched a handful of long-standing vulnerabilities in their publicly exposed Windows server products that could have been used to create a devastating internet flashworm. But that didn't happen, nor did it happen. Last month or the month before, and I doubt it will happen next month. For some time, we've seen serious vulnerabilities publicly exposed in Cisco Edge border routers that could have been leveraged to do the same thing, but that's never occurred. My own theory, based upon watching the use of vulnerability exploits for many years, is that disrupting or taking down the internet is not profitable. What is highly profitable is breaking into an organization, exfiltrating their data, and then extorting payment from the breached organization in return for that data's deletion. So, that is what has been going on. And there's every reason to believe that's what will continue to go on. Today's AI will likely serve as nothing more than an accelerant poured over the present status quo. I don't think it's going to see the world change. So, I just expect that we're going to be seeing more of the same from the malicious use of AI rather than anything apocalyptic. And once the products of defensive AI finally make their way into enterprise networks, such AI-enhanced intrusions, those too will likely begin to dry up. Destroying the global internet, which is directly facilitating attackers' revenue stream, would be entirely self-defeating. Again, it has already been possible entirely without AI to wreak tremendous chaos on the internet, and it has never happened. So, my final take on all this is that we're going to stumble and bumble forward as we always do. Yes, there'll be bumps and mistakes along the way, but we're going to be fine. Change is always a challenge, and there has never been a change more sweeping than AI. Doug wrote, I'm tired of hearing that this is just the latest automobile or telephone or internet bringing disruption into our lives. Well, Doug may be tired of hearing that somewhere, but that's certainly not something that's ever been said here. I have absolutely no doubt, and I know, Leo Laporte on the same page here as I am, that this generation of artificial intelligence will prove to be the biggest change we have ever experienced during our lives. Yeah. We have actually created a machine that knows everything. It contains all knowledge. That can be intoxicating, intimidating, thrilling, and incredibly useful. I expect that there will likely be many mistakes and missteps made. You know, as I said near the top of this, I'm glad that the AI execs are publicly freaked out and frightened and that they fine, pause for a while. You know, pause yourself. Yeah, there's no hurry. Yeah, exactly. God knows you're turning out a new model every day. So fine, take your time. Work out the training, work out the alignment, work out the guardrails, monitor and control agentic AI. You know, we didn't have it six months ago. This is all still very new. And Andrew Ng was correct to say that these problems will have solutions. The fact that we don't have them yet doesn't at all mean that they're impossible for us to engineer and get. So I fully expect that everyone listening to this podcast today will have the opportunity to live out their full natural lifetimes without AI bringing it to a premature end.

Nothing to worry here. Move on. You know, I think the question for you, Doug, would be: do you

acknowledge that there are benefits to be achieved from this technology? And I think that that's the thing Steve and I both are saying is that we can see significant, I already see significant benefits, and I see many, many more coming down the road. If you see benefits to this, yes, there's also potential harm, but it would seem foolish to say, well, because of this potential harm, we got to stop because we don't want to take the risk and lose the benefits. And I think there are going to be some significant benefits. I also think it'll be, regardless, highly disruptive. But so every technology is. So was the steam engine. So is the industrial era. So was the locomotive. I mean, so was the automobile. You could. Honestly, make a very strong argument. A million people a year are killed and injured by automobiles that we should never have allowed the automobile to exist.