a16z Podcast

Fei-Fei Li & Yunzhu Li on Why Robots Learn in Simulated Worlds

Fei-Fei Li & Yunzhu Li· Co-founder & CEO of World Labs; Co-founder of SceniX at World Labs
·~42 min·English·a16z
RoboticsMultimodalTrainingAI Infrastructure
TL;DR

World Labs and SceniX argue the path to robots that work runs through simulated worlds: because real-world robot data is scarce, a real-to-sim-to-real loop manufactures the counterfactual coverage, reliability, and fast evaluation that real data alone can't.

01Core Mental Model

Spatial Intelligence Is a World You Act In, Not Just See

World Labs defines spatial intelligence as AI that can generate, understand, reason about, and act within spaces — which makes robotics the natural endpoint of computer vision, not a side quest.

spatial intelligence is about um creating AI that has the ability to generate uh understand reason with and interact with spaces whether it's physical or virtual

— Fei-Fei Li, a16z Podcast
Key Insight
The reframing does real work: by defining intelligence as the ability to act in space, not just recognize it, World Labs moves the goalposts from perception — a far more mature capability — to interaction. That is why acquiring a robotics team reads as continuity rather than a pivot: robots are simply where 'act within spaces' has to land.
02The Core Problem

Robotics' Bottleneck Isn't the Brain — It's Data

Unlike language models, robots have no internet-scale corpus of how the physical world responds to actions, so the field cannot simply scale its way out of the data shortage.

we just do not have cannot possibly have enough real world data for that

— Fei-Fei Li, a16z Podcast
Key Insight
The asymmetry is the entire thesis. LLMs got a free lunch — humanity already wrote the internet. Robotics has no equivalent record of how to physically do things, so every downstream bet World Labs makes follows from accepting that the data will not come from the real world and has to be manufactured elsewhere.
03The Mechanism

The Fix Is a Real-to-Sim-to-Real Loop

SceniX maps a real environment into an aligned digital twin, generates training and evaluation data there at scale, then ships the resulting policy back to the real robot — and feeds new real data back in.

such that we can replace all the data all the evaluation we need in the real environment by using the data that can generate at a scalable way in our digital world

— Yunzhu Li, a16z Podcast
Key Insight
The clever move is treating the simulator as a data factory, not a testbed. Most teams use sim to check a policy; the real-to-sim-to-real loop uses it to manufacture the training and evaluation data itself. That only holds if what happens in sim also happens in reality — the alignment claim everything else rests on.
04The Deep Reason

Simulation's Superpower: Counterfactual Reasoning

Real data can only show what did happen, but simulation lets a robot play out events that haven't or can't happen — the counterfactuals it needs to learn how to act.

There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning

— Fei-Fei Li, a16z Podcast
Key Insight
This is the deepest point in the interview and it answers the standard objection — that a simulator always drifts from reality. The counter isn't 'our sim is perfect'; it's that real data only ever captures the path taken, while robots fail on the rare and dangerous cases nobody recorded. Waymo's own bet — more simulation than real driving — is the existence proof, and cars, as they note, are the simplest robots.
05The Payoff

Two Payoffs Real Data Can't Give: Reliability and Efficiency

Simulation buys reliability through controlled, systematic coverage of the state space, and efficiency by speeding robots past the human-speed ceiling of tele-operation.

simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency

— Yunzhu Li, a16z Podcast
Key Insight
Splitting the payoff in two reframes simulation from a cost-saver into a capability-unlocker. Reliability comes from controlled variation — you can prove which slice of the state space you covered, which real-world collection can never claim. Efficiency comes from breaking tele-operation's human-speed ceiling. Neither is reachable by simply collecting more real data.
06The Overlooked Half

Evaluation Is the Silent Bottleneck in Robotics

Because you can only improve what you can measure, robotics' real tax is evaluation — which in the real world runs orders of magnitude slower than the iteration loops that made LLMs compound.

the iteration speeds is multiple orders of magnitude slower than iterations of those language models

— Yunzhu Li, a16z Podcast
Key Insight
The overlooked claim is that evaluation, not training, is where robotics loses to LLMs. LLM teams iterate quickly; robots iterate at the speed atoms move through space. Simulation doesn't just cut cost — it restores the tight, fast eval loop that let LLM progress compound, which is why the team stresses the part people 'often overlook.'
07The Deployment Path

Robots Climb From Structured Worlds to Unstructured Ones

Robotics has always advanced from fully structured environments to semi-structured to unstructured, so the pragmatic play is to win warehouses and restaurants before attempting the home.

it has always followed the trend from going from fully structured environments into semiructured environments and then into unstructured environments

— Yunzhu Li, a16z Podcast
Key Insight
The progression is a risk-management strategy dressed as a taxonomy. Robustness equals coverage, and you can only cover a world you partly control — so starting in warehouses isn't timidity, it's the only place today's coverage is achievable. It also quietly demotes the home robot from a near-term product to a distant grand challenge.
08The Contrarian Take

Humanoids Are the Hardest Problem, Not the Right First One

A general humanoid body is what evolution built for survival across unstructured worlds — general but best at nothing — which makes it the hardest robotics target, and not necessarily the right one to start with.

this unstructured environment and a generalized body is actually the hardest problem to solve. It's not necessarily even the right way to solve the problem

— Fei-Fei Li, a16z Podcast
Key Insight
The contrarian move is separating 'hardest' from 'right.' Evolution optimized the human body for survival across unstructured worlds — generality bought at the price of never being best at anything. A business doesn't need survival; it needs a narrow task done reliably, which argues for specialized bodies now. Fei-Fei's coda — the human brain runs on 30 watts, and even LLMs don't match it — is the reminder that measured optimism, not humanoid hype, is the hard discipline.