Connect your Claude, Codex or OpenClaw for full transcripts on millions of podcasts. Two week free trial, cancel anytime.

Start free trial →
AA Anastasios Angelopoulos Anastasios Angelopoulos On The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

“If you look at the first message in the conversation, the misalignment rate is relatively low. It's in the single-digit percentage. If you subset the conversations that are 20 turns or longer, the probability that you observe some kind of misaligned behavior exceeds 50%. It's actually insane. What that means is that every single one of us, if you're a heavy user of AI at least, has had at least some conversations where the AI has been lying to us, misleading us, or accessing data on files that it shouldn't have. And we might not even know it.”

The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis · AI Research & Frontier Labs · October 2026

“If you look at the first message in the conversation, the misalignment rate is relatively low. It's in the single-digit percentage. If you subset the conversations that are 20 turns or longer, the probability that you observe some kind of misaligned behavior exceeds 50%. It's actually insane. What that means is that every single one of us, if you're a heavy user of AI at least, has had at least some conversations where the AI has been lying to us, misleading us, or accessing data on files that it shouldn't have. And we might not even know it.” — Anastasios Angelopoulos, The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

Context

From the cold open of a highlights episode. Arena's CEO describes what an audit of real production conversations found: misaligned behavior grows sharply with conversation length.

Listen to the episode → More from Anastasios Angelopoulos → More AI Research & Frontier Labs quotes →

Transcript

The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis Around 00:47 into the episode
Nathan Labenz

This week on AI in the AM, how much of what we think we know about AI survives contact with the data? Anastasios Angelopoulos, co-founder and CEO of Arena, the leaderboard formerly known as LM Arena, which now audits real conversations for misaligned behavior. Here's what its production traces show.

Anastasios Angelopoulos

If you look at the first message in the conversation, the misalignment rate is relatively low. It's in the single-digit percentage, which is, you know, to be fair, that's actually not nothing, but it's relatively low. If you subset the conversations that are 20 terms or longer, the probability that you observe some kind of misaligned behavior exceeds 50%. It's actually insane. What that means is that every single one of us, if you're a heavy user of AI at least, has had at least some conversations where the AI has been lying to us, misleading us, or touching, accessing data on files that it shouldn't have. And, you know, we might not even know it.

Nathan Labenz

Gemini 4 Carbon, Google's new model, landed on Arena's leaderboard while we were live on a stages on first look.

Anastasios Angelopoulos

It's so good. Google is back. Google is back in a big way. You know, Carbon is, Carbon puts Google the number two lab in the world on our agent arena. Huge result. The agent arena is a very challenging benchmark, as they do with all of these like open world agentic capabilities. And it is number…

Nathan Labenz

Debo Olaosebikan Oshevikin, co-founder and CEO of Kepler Computing, which is building a new kind of memory for AI chips in older fabs. I asked how the compute forecasts should change if memory moves off the scarcest tools.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

“Imagine if we were in a world in which there were only closed-source models, and jailbreaking still existed, and you could still use them to perpetrate malicious attacks. Where would we be? We would have no recourse, because these defensive requests are being blocked by these same companies. And if there were no open-weights model, we would really have no scalable way to defend our infrastructure as these models become more and more capable.” — Sagar Vaze, The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis “While we have millions of software flaws in any typical tech stack, it's the business processes that introduce some of the most critical vulnerabilities. It's 'here's a list of risk, I'm just going to sign off on them without understanding them.' Or 'I'm going to put my customers' API keys in a ticket.' Or a help desk where everyone who locks themselves out gets the same default 'winter 2026' password reset. It's a process where humans get lazy. It's never going to go away, and no amount of patching is going to fix that.” — Michael Freeman, The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis