Google DeepMind: The Podcast

Koray Kavukcuoglu on Why AGI Has No Finish Line

Koray Kavukcuoglu· Chief AI Architect at Google; SVP, Google DeepMind at Google DeepMind
·~47 min·English·Google DeepMind
LLMAgentsTrainingAI Safety
TL;DR

Google DeepMind's Koray Kavukcuoglu argues there is no test for AGI — only products, trust and human usefulness that compound release by release — while a slice of Gemini's compute already runs under agent control, though the full self-improvement loop stays far off.

01Core Mental Model

There Is No Test for AGI — Only a Ladder You Climb

<strong>AGI is not a benchmark you pass</strong> but a line that keeps moving; what matters is whether you trust the models a little more with every release.

We don't have a test for AGI. We'll never have a test for AGI.

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
By making usefulness part of the definition of intelligence, Kavukcuoglu quietly disarms the 'did we reach AGI yet' debate — if a system is not useful to you, he argues, calling it intelligent is empty.

02Strategy

Products Are the Stepping Stones to AGI

AGI is only the north star; <strong>products are the concrete stones</strong> that tell you which problem to solve next — so building AGI without them, he says, will not happen.

trying to build AGI without products in the loop. I don't think it's going to happen.

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
The stepping-stone framing is also a competitive bet: by wiring Gemini into Maps, Gmail and Docs, DeepMind takes on a harder integration problem than a narrow lab would — but earns a feedback signal no benchmark can give.

03Capability

Spiky Intelligence, and the Restaurant Test

Today's models are brilliant at hard reasoning yet clumsy at booking a restaurant, and <strong>generality means you do not get to choose which tasks matter</strong>.

I mean they're much less spiky than what they were but still they are spiky.

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
The mundane 'book a restaurant' task is not a distraction from intelligence — in his framing it is the test of it, because a truly general system cannot declare the boring tasks out of scope.

04Mechanism

Intelligence Is Transfer, Not a Pile of Narrow Wins

Solving many narrow tasks is not intelligence; <strong>intelligence is transfer</strong> — generalizing into problems the model was never trained on.

Intelligence is the generalization over these. Intelligence is transfer from these into something that you haven't seen before.

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
He is candid that transfer to genuinely unknown problems is still 'very very early' — the excitement is less about today's results than about the one capability that, if it scales, makes the narrow-task era obsolete.

05Research Philosophy

Anchor on Human Data — His Answer to the Bitter Lesson

Self-generated data lets a model explore beyond humans inside closed games, but <strong>useful AGI stays anchored on human data</strong> — his 'yes and no' to removing it.

I think it is critical that we anchor ourselves on human data because we need models that are useful for us in our world that we can use as tools

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
Against the AlphaZero reading of the bitter lesson, he reframes human data not as a crutch that caps a system but as what keeps it aimed at our world: for a tool you can rely on, the human anchor is the point, not a limitation.

06The Frontier

Recursive Self-Improvement: Pockets, Not a Button

<strong>A slice of Gemini's compute already runs under agent control</strong>, running experiments — but the full self-improvement loop, he says, is still a bit far away.

it's that self-improvement loop we are a bit far away from that

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
The admission cuts against the hype while confirming the direction: DeepMind already trusts agents with real experiment compute, so the question is no longer whether RSI starts but how far the automated 'pockets' spread.

07Safety

Keep the Thoughts Pristine

DeepMind deliberately <strong>puts no reward on the chain of thought</strong>, so an agent's reasoning stays an unfiltered, monitorable window on what it is about to do.

We pay a lot of attention not putting rewards on those thoughts so that the agent's thoughts are in a way unfiltered and at execution time we can monitor those

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
This is the concrete mechanism under the abstract word 'alignment': safety here is an architecture decision — keep reasoning legible in natural language — not a filter bolted on after training.

08Safety Mindset

From the Science of AGI to the Engineering of It

Once coding and tools let agents leave the sandbox, building AGI became engineering — and <strong>you don't build anything that is not safe</strong>.

we moved from science of researching AGI into building AGI which is a lot more of an engineering activity. You don't imagine building anything that is not safe.

— Koray Kavukcuoglu, Google DeepMind: The Podcast
Key Insight
Framing the frontier as engineering rather than science moves safety from an outside check to part of the build itself — you no more ship an unsafe model than open an untested bridge, which is why air gaps, sandboxes and government testing sit inside the process, not around it.