NVIDIA Fireside Chat

Gilad Shainer on why the AI factory needs four networks, not one

Gilad Shainer· SVP of Networking at NVIDIA at NVIDIA
·~30 min·English·NVIDIA
AI InfrastructureGPUInferenceTraining
TL;DR

NVIDIA's networking chief argues an AI factory needs four purpose-built networks — not off-the-shelf Ethernet — to turn a datacenter full of GPUs into a single supercomputer.

01Core Mental Model

Four Networks, Not One

An AI factory is not wired by a single network — it needs four separate, purpose-built ones, each solving a problem the old datacenter never had.

And obviously there is no one infrastructure that fits all.

— Gilad Shainer, NVIDIA Fireside Chat
Key Insight
Shainer is quietly retiring the idea of 'the datacenter network.' A traditional datacenter had one network, maybe two counting DCI; an AI factory has four plus a secure access layer. That is not a scaling story — it's a claim that AI broke the single-fabric assumption datacenters were built on.
02Scale-Up

Wiring Many GPUs Into One

The scale-up network exists to fuse as many as 1,152 GPUs into a single GPU unit, and it carries roughly 100 times the bandwidth of traditional compute.

but that scale up network is at least 100 x the bandwidth of what we've traditionally seen in compute.

— Alan Welkel, NVIDIA Fireside Chat
Key Insight
The number that matters here isn't the GPU count, it's the 100x. Bandwidth that extreme is only worth building if the goal is to make many chips act like one — which is why NVLink is a distinct, purpose-built fabric and not just faster Ethernet. The wiring is the product.
03The Wiring Decides

Server Farm or Supercomputer

Take the exact same GPUs and the network alone decides whether you get a supercomputer or just a pile of servers.

If you if you connect them with an off the shelf Ethernet network, you just build a server farm. No more to that.

— Gilad Shainer, NVIDIA Fireside Chat
Key Insight
This is the load-bearing claim of the whole interview: value has migrated from the chip to the fabric between chips. Ethernet built for single servers delivers bandwidth but not jitter-free synchronization — so the same silicon that could be a supercomputer degrades into an ordinary server farm the moment the network is generic.
04Cadence

Two Generations at Once

The generational cadence dropped from one every three-to-four years to something new every single year, forcing teams to build two generations in parallel.

So you're already working on two generations in parallel. And you need to understand what are the technologies that are going to come next.

— Gilad Shainer, NVIDIA Fireside Chat
Key Insight
A yearly cadence quietly rewrites who can compete. When you must ship a new GPU, switch, and SuperNIC every year while already designing the one after, a strong research arm stops being a nicety and becomes the entry fee — the barrier is organizational stamina, not any single chip.
05Power Ceiling

Power Is the Real Limit

Because power caps how much compute you can install, and the optical network burns nearly a tenth of it, moving the optics onto the switch package saves about five times the power.

It's not just saving power almost five x less power and optical network. We're also seeing ten x increase in the time or mean time between interrupts.

— Gilad Shainer, NVIDIA Fireside Chat
Key Insight
Notice the power savings come with a reliability bonus nobody asked for. Shortening the distance data has to travel doesn't just cut watts — fewer pluggable parts means ten times longer between interrupts. In a factory where power is the hard ceiling, an efficiency move that also raises uptime is the rare free lunch.
06Performance

Zero Jitter, Zero Collisions

The whole design target is the hardest one to hit — zero jitter — and in the field it showed up as 95% effective bandwidth with zero collisions across 100,000 GPUs.

And in my eyes, again, Alan, it's it's zero jitter. It's the most complicated things to achieve.

— Gilad Shainer, NVIDIA Fireside Chat
Key Insight
Jitter sounds like a performance nicety; Shainer treats it as the foundation of multi-tenancy. Zero collisions means no congestion, and no congestion means one customer's workload physically cannot steal performance from another's — so the boring-sounding jitter metric is actually what lets you sell an SLA on a shared AI cloud.
07Division of Labor

No Single Box Can Do It

You can't build the AI network in one device because free traffic-spraying and in-order delivery cannot live in the same box — the switch and the NIC each take half the job.

You cannot do both at the same place. And that's why you cannot implement an AI network just on a switch.

— Gilad Shainer, NVIDIA Fireside Chat
Key Insight
This is the technical reason the four-network story can't be shortcut. Spreading traffic across every port for bandwidth and guaranteeing packets arrive in order are contradictory jobs — so the intelligence has to be split, switch and SuperNIC, and co-designed end to end. It's an argument for owning the whole fabric rather than buying parts.