KK Koray Kavukcuoglu Koray Kavukcuoglu SVP of Google DeepMind and Chief AI Architect at Google, running the organization day to day and overseeing Gemini model development, frontier AI research, and...

“What has happened in the history of machine learning over years is playing out over weeks.”

Google DeepMind: The Podcast · AI Research & Frontier Labs · November 2026

“What has happened in the history of machine learning over years is playing out over weeks.” — Koray Kavukcuoglu, Google DeepMind: The Podcast

Context

Fry asked whether anyone can tell who is winning when benchmarks are saturated or contaminated. He says benchmarks now exist to expose capability gaps, and saturation is just what progress looks like before the next benchmark replaces the last.

Transcript

Google DeepMind: The Podcast Around 36:31 into the episode
Koray Kavukcuoglu

…in ourselves to make sure that we build mitigations that we think are going to produce a model that is robust and safe enough for people to use, like without. Unintended side effects. All these things are there because we care about the technology and the technology being safely in people's hands.

Hannah Fry

…Can you even tell who's winning? I mean, I guess this is really a question about benchmarks, right? Lots of these benchmarks are becoming saturated, they're becoming contaminated by the training data. I mean, everyone's talking about it being a race, but can you even tell who's in the league?

Koray Kavukcuoglu

…to create environments, I'm going to collect data, and then I'm going to train my model with the expectation that it's going to do better on the task. That's why gradually we see all these benchmarks getting saturated because the models are getting better in many different areas. That is what it should be, that's what progress looks like. And then, of course, then we explore new benchmarks. What has happened in the history of machine learning over years is playing out over weeks.

Hannah Fry

But then, do you think this will keep going? That it's just you saturate the benchmarks, you have new benchmarks, you saturate those benchmarks, and so on and so on, and so on.

Koray Kavukcuoglu

That's how we train the models, don't we?

Hannah Fry

Yeah, I suppose. Right,

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Google DeepMind: The Podcast