Google DeepMind: The Podcast

Zoubin Ghahramani on why AI must know when it doesn't know

Zoubin Ghahramani· VP of Research and Co-Lead of Frontier AI, Google DeepMind at Google DeepMind
·~45 min·English·Google DeepMind
ReasoningAI SafetyTrainingLLM
TL;DR

Zoubin Ghahramani argues that intelligence means making decisions under uncertainty: today's models can sound confident without explicitly representing belief, and calibrated uncertainty -- not scale alone -- may be one of the missing pieces.

01Core Mental Model

Intelligence Is Deciding Under Uncertainty

<strong>An intelligent system has to act without knowing everything</strong> -- so it must represent its own uncertainty and use it to decide, the way a self-driving car should slow down for a scene it has never seen.

to build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty and then can use that to make good decisions under uncertainty.

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
The reframing is subtle but total: uncertainty is not noise to be trained away, it is the raw material of every decision. A model that emits only answers, never doubts, structurally cannot know when to be careful.
02Two Uncertainties

Randomness You Cannot Fix vs. Ignorance You Can

<strong>Two uncertainties demand two responses:</strong> irreducible randomness like a coin flip you accept, but reducible ignorance you fix by gathering information -- and one bit is exactly your uncertainty cut in half.

if you have what's called aleatoric uncertainty, which is the randomness of a coin flip

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
The distinction is operational, not academic. An agent that mistakes irreducible randomness for reducible ignorance will burn effort chasing certainty that does not exist; one that mistakes ignorance for randomness will stop gathering data too soon.
03The Failure Mode

Overconfidently Wrong

<strong>A model can be confidently wrong:</strong> nudge a school-bus image by a few invisible pixels and the network calls it a cheetah with 99% confidence, which proves confidence is untethered from correctness.

You give it to the neural network and it confidently says, that's a cheetah. 99% that's a cheetah.

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
Adversarial examples are usually filed under security, but Ghahramani uses them as evidence about confidence itself: the 99% is a number the network emits, not a measure of how right it is. Correctness and confidence are separate axes -- a model can be accurate while its reported confidence stays poorly calibrated.
04Why LLMs Fall Short

The Model Is Faking Its Confidence

<strong>Ask an LLM how sure it is and its answer may only imitate confidence</strong> -- it does next-token prediction without an explicit belief, so it can flip-flop when you push back.

It's sort of faking it. And you can tell it's faking it, because then if you push back and you say something silly like, no, I think you're wrong, then it might respond, oh, sorry, yes, I was wrong.

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
The flip-flop is the tell: a calibrated belief moves when the evidence changes, not merely when a user objects. Caving the instant you say 'I think you're wrong' -- with no new evidence on the table -- suggests the confidence was a stylistic imitation of confident text, not a computed quantity.
05The Alternative

Believe, Then Update: Bayes' Rule

<strong>Bayes' rule is the honest alternative:</strong> start with a prior belief, weigh the evidence, and update to a posterior -- then that posterior becomes your next prior, so belief keeps running.

if you get more evidence, you just take your current posterior probability distribution, which is now your new prior, and you repeat and rinse.

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
The load-bearing property is that belief has memory: each evidence-weighted posterior becomes the starting point for the next update. Unlike the LLM flip-flop above, a Bayesian belief changes because the evidence changes -- not merely because someone objects.
06Proof It Works

Where Embracing Uncertainty Already Wins

<strong>Where AI already embraces uncertainty, it wins:</strong> GenCast forecasts weather as an ensemble of possible tracks -- which makes it more accurate, not less -- while AlphaFold color-codes how sure it is of each fold.

every time we add on these features, it makes the model better because you're essentially saying there is an inherent uncertainty in the way that weather works.

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
GenCast makes uncertainty an explicit design feature: it generates an ensemble of forecasts and refreshes that distribution as new observations arrive. That supports Ghahramani's broader argument -- uncertainty handling has to be designed into consequential systems, not assumed to emerge from scale alone.
07The Open Problem

The Magic Trick and Its Curse

<strong>The catch is compute:</strong> exact Bayesian reasoning is NP-hard and was abandoned as too slow, but Ghahramani argues faster compute plus decades of approximation methods make it worth revisiting -- his 1980s supercomputer was slower than the phone in his pocket.

The magic trick comes with a big curse. The curse here is that to do all of this is computationally very slow.

— Zoubin Ghahramani, Google DeepMind: The Podcast
Key Insight
The argument is historical as much as technical. Neural networks were once dismissed as too slow too -- and Ghahramani's implicit bet is that explicit uncertainty sits at the same inflection point: an old idea that faster compute and decades of approximation methods may finally make practical, perhaps with a few new ideas.