Koray Kavukcuoglu on Why AGI Has No Finish Line
Google DeepMind's Koray Kavukcuoglu argues there is no test for AGI — only products, trust and human usefulness that compound release by release — while a slice of Gemini's compute already runs under agent control, though the full self-improvement loop stays far off.
There Is No Test for AGI — Only a Ladder You Climb
<strong>AGI is not a benchmark you pass</strong> but a line that keeps moving; what matters is whether you trust the models a little more with every release.
We don't have a test for AGI. We'll never have a test for AGI.
Products Are the Stepping Stones to AGI
AGI is only the north star; <strong>products are the concrete stones</strong> that tell you which problem to solve next — so building AGI without them, he says, will not happen.
trying to build AGI without products in the loop. I don't think it's going to happen.
Spiky Intelligence, and the Restaurant Test
Today's models are brilliant at hard reasoning yet clumsy at booking a restaurant, and <strong>generality means you do not get to choose which tasks matter</strong>.
I mean they're much less spiky than what they were but still they are spiky.
Intelligence Is Transfer, Not a Pile of Narrow Wins
Solving many narrow tasks is not intelligence; <strong>intelligence is transfer</strong> — generalizing into problems the model was never trained on.
Intelligence is the generalization over these. Intelligence is transfer from these into something that you haven't seen before.
Anchor on Human Data — His Answer to the Bitter Lesson
Self-generated data lets a model explore beyond humans inside closed games, but <strong>useful AGI stays anchored on human data</strong> — his 'yes and no' to removing it.
I think it is critical that we anchor ourselves on human data because we need models that are useful for us in our world that we can use as tools
Recursive Self-Improvement: Pockets, Not a Button
<strong>A slice of Gemini's compute already runs under agent control</strong>, running experiments — but the full self-improvement loop, he says, is still a bit far away.
it's that self-improvement loop we are a bit far away from that
Keep the Thoughts Pristine
DeepMind deliberately <strong>puts no reward on the chain of thought</strong>, so an agent's reasoning stays an unfiltered, monitorable window on what it is about to do.
We pay a lot of attention not putting rewards on those thoughts so that the agent's thoughts are in a way unfiltered and at execution time we can monitor those
From the Science of AGI to the Engineering of It
Once coding and tools let agents leave the sandbox, building AGI became engineering — and <strong>you don't build anything that is not safe</strong>.
we moved from science of researching AGI into building AGI which is a lot more of an engineering activity. You don't imagine building anything that is not safe.