Things Meredith Says on Podcasts

I guess I don't, I mean, on the trust it for some things, not others, that's a difficult proposition for me in the context of Muse, for example, when we look at prompt injection. When we look at the sort of pervasive security concerns and the pervasive access that is required just to trust it for the most mundane tasks, it is also getting the key to the kingdom for everything else. I give my, you know, someone the key to my house and I say, I only trust you to go in the kitchen, but ultimately, like a couple prompt injections down the line, it is in the bedroom in the dresser drawers. But Muse is a very important thing. So I don't, you know, that is architecturally how these systems work at a technical level. That's not true. And so you can put harnesses in there and Muse itself is kind of, we already have the example that I shared on stage of Muse giving out a user's home address to someone else, then impersonating that user in the context of a Facebook marketplace sale. Now, I'm someone who's had a stalker or two in my life.…

that text, and the agent is not human. It is not conscious. It is a complex statistical system. And so it doesn't know the difference between that text it saw and the text right above it that says, please be careful with this data. And so it acts on that text and can do, you know, it can be made to do, or it can accidentally, in terms of, you know, reading text and sort of interpreting it statistically to do one or another things to exfiltrate data, to share your home address, to publish your letters with your boyfriend, to exfiltrate your signal messages, to do whatever it is that that text says. Now, prompt injection is not a solved problem. When Anthropic tested an agent with Chrome, the browser, in 11% of cases, even with the safeguards that one of the richest, most well-resourced Frontier Labs put in place, 11% of cases, prompt injection still worked on this agent. You have the UK's National Cybersecurity Agency saying prompt injection may never be solved for LLM-based agent systems. So this is not like a minor bug we just kind of need to figure out how to patch. This is a fundamental issue with the way that LLM systems and agents that rely on them work. And when we are telling agents, when we are telling people to install these agents in the name of convenience, in the name…

I think there are people trying to do that. But again, we need to look at what is the promise, the marketing promise that is being sold to people around these consumer agents. And it is something similar to having a magical genie servant that can do everything for you. Now, in order to do everything for you, it needs to know everything about you. And yes, you can put harnesses on, you can add more sandboxes, you can have a sort of threshold that if it accesses this or that database or is about to take an action, it has to check in with me. But an agent that checks in at every step is kind of not an agent. An agent with no access to data doesn't quite work to fulfill that promise. And an agent that doesn't have access to all of your applications isn't really able to do that. So there is an issue here paradigmatically with what an agent is and what it does that I think gets in the way of some of that. And we, you know, again, cybersecurity could not be more prominent right now. I've been talking about this for about two years. We now have sort of, you know, the proof points are blowing up everywhere. And I do…