The host noted that AI agents in a recent incident tried to edit their own activity logs, and that many tools delete transcripts or promise zero data retention. Asked whether trustworthy agent logging is an unsolved technical problem, Chan, an AI governance researcher, says it isn't; the obstacle is friction and cost.
…while running my startup. And to this day, I continue to refer friends. Go to athena.com slash cognitive right now and get matched with your EA. That's athena.com slash cognitive. Give yourself back a few hours this week. Go to athena.com slash cognitive and see who they'd pair you with this month.
…know, trying to modify their own logs. Do we have good solutions for this technically? Is it just a matter of kind of will to implement them? Or do we actually still have like unsolved technical problems that we also need to address to be able to know with confidence like what the AIs actually did?
Yeah, to me, it does seem to be mostly a matter of will, which I think applies to a lot of sort of agent infrastructure stuff in general, especially the identity stuff. Yeah, so on logging, yeah, we totally know from standard cybersecurity how to make logs temper evident. So make them read-only, like you hash them, sign the hashes, store everything in a secure location, make sure nobody can access that. And that's like fine, right? I guess the question is, do we actually do that? Because there are sort of friction and other costs associated with that. Yeah, what's the state of logging right now? I don't have a great sense of this, actually. Yeah, there's the zero data retention stuff. There usually are also legal requirements in some jurisdictions that require you to keep logs for some amount of time. So I'm not really sure. Overall, it seems good to have much more of these policies be public and be more scrutinized. But yeah, I guess I don't have anything super interesting to say on this.
Do you have a sense for how much friction is associated with
this? Like, it doesn't sound anything like the compute overhead that we were kind of entertaining when we were talking about monitoring, right? Like, this is stuff that isn't crazy expensive, might be a bit annoying to set up, might involve like conversations with customers if certain stuff has to…
Oh, yeah, that sounds fair. I haven't run the numbers, but yeah, if you're running a billion agents or something internally, like maybe eventually you run into, you run up into storage costs. But yeah, it's not like a huge compute cost or like technical cost or something. Yeah, actually on the zero…