KK Koray Kavukcuoglu Koray Kavukcuoglu SVP of Google DeepMind and Chief AI Architect at Google, running the organization day to day and overseeing Gemini model development, frontier AI research, and...

“We pay a lot of attention not putting rewards on those thoughts so that the agent's thoughts are in a way unfiltered. And at execution time, we can monitor those because every action that the agent will take will actually first manifest in its thoughts.”

Google DeepMind: The Podcast · AI Research & Frontier Labs · November 2026

“We pay a lot of attention not putting rewards on those thoughts so that the agent's thoughts are in a way unfiltered. And at execution time, we can monitor those because every action that the agent will take will actually first manifest in its thoughts.” — Koray Kavukcuoglu, Google DeepMind: The Podcast

Context

Fry asked how Google stops agents from finding ways around their constraints. He describes monitoring the chain of thought alongside actions and side effects, and calls keeping those thoughts untrained-on critical for safety. It comes the same week OpenAI's leadership said chain-of-thought monitoring is getting harder.

Transcript

Google DeepMind: The Podcast Around 29:49 into the episode
Koray Kavukcuoglu

…can do in the world. The measurement of research is very objective, it has to be. But why someone goes and researches a topic that until you actually prove that it is true, it is not known, is a very subjective phenomenon. And that is you, your creativity, and your motivation as a researcher.

Hannah Fry

…were researching was reinforcement learning for years. And presumably, during that time, you spotted algorithms coming up with all sorts of ingenious ways to work their way around the constraints that you try and place on them. How do you mitigate against those now that we're in a world of agents?

Koray Kavukcuoglu

…from what you intended to do, you can actually interject. And one of the most important things for this is designing architectures that are monitorable. Like, right now, our systems have a chain of thoughts, right? They're not visible to the end user. These are like, I mean, literally, the model's thoughts. And those thoughts being pristine, they are being untouched, is very critical for safety. We pay a lot of attention not putting rewards on those thoughts so that the agent's thoughts are in a way unfiltered. And at execution time, we can monitor those because every action that the agent will take will actually first manifest in its thoughts. Together with the actions, we monitor the actions of the agent, the environment, what it does, the side effects on the environment, and the agent's thoughts to build a detector of is the agent still executing on what the user intended.

Hannah Fry

But it's not just agent, I mean, agents as well. And I know a lot of your earlier work was about sort of creating teams of agents to achieve a particular objective. And now we're really in an era where we're seeing teams of agents working together to sort of do slightly more underhand things.

Koray Kavukcuoglu

Yes.

Hannah Fry

How do you monitor those?

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Google DeepMind: The Podcast