NVIDIA fireside chat

Ian Buck and Sachin Katti on why the data center is the new computer

Ian Buck & Sachin Katti· infrastructure leaders at NVIDIA and OpenAI
·~20 min·English·NVIDIA
AI InfrastructureGPUInferenceTraining
TL;DR

NVIDIA's Ian Buck and OpenAI's Sachin Katti describe how frontier AI has turned the whole data center into the unit of compute: a co-designed compute token factory where power, cooling, networking, and even the model tuning its own inference are all one design problem.

01Core Mental Model

The Data Center Is the Computer Now

The unit of AI compute has climbed from a single GPU to an entire building, so what you design end to end is no longer a chip but a compute token factory.

You have to design a compute token factory from end to end.

— Ian Buck, NVIDIA fireside chat
Key Insight
The reframing moves the optimization target off the chip. Once the whole building is the machine, power, cooling, and networking stop being facilities overhead and become first-class parts of the compute design.
02Origin Story

The Day One Box Wasn't Enough

Trillion-parameter models broke the old unit of a single eight-GPU server and forced NVIDIA to spread one machine across a rack of 72 GPUs linked by NVLink.

We put eight GPUs in one node, and that was kind of the unit of compute was a single server

— Ian Buck, NVIDIA fireside chat
Key Insight
The rack of 72 is not just a bigger server; NVLink lets those GPUs act as one, which is how NVIDIA made a trillion-parameter model practical to train and serve. Buck calls the shift an existential moment because the whole engineering stack had to change with it.
03Design Insight

Schedule by Power, Fit More GPUs

Different jobs draw power differently, so a scheduler that knows each job's draw can load-balance the electricity and pack far more GPUs into the same power budget.

you can get upwards of 30 or 40% more GPUs in the data center, but you got to design that from the beginning

— Ian Buck, NVIDIA fireside chat
Key Insight
The gain does not come from faster chips but from ending the habit of provisioning every rack for a peak it rarely hits. Buck's condition, design it from the beginning, is the catch: the smart power fabric has to be planned up front, not bolted onto a data center built in silos.
04Beyond the Data Center

A Data Center That Could Steady the Grid

Fully built out at eight gigawatts, the Ohio site would be a large share of the state grid, so Katti argues a co-designed data center could ramp jobs down during spikes and up during gaps to help keep the grid smooth.

if designed right, this can help stabilize the grid. In Ohio, whenever there's spikes, we can go down on our jobs. Whenever there's gaps, we can actually go up so that we can keep the grid smooth.

— Sachin Katti, NVIDIA fireside chat
Key Insight
This flips the usual framing that AI data centers only strain the grid. The lever is that training and batch jobs are interruptible: a co-designed gigawatt-scale load can ramp with the grid instead of fighting it, acting like a buffer rather than a constant strain.
05The Striking Result

The Model That Optimizes Itself

OpenAI's Astra, moved unchanged from Blackwell to Vera Rubin, delivered 3x the throughput out of the box, then Astra agents tuned Astra's own inference on Rubin for another 2x over a weekend.

It's a recursive loop. The model is figuring out how to optimize its own inference, serving on Rubin chips and over 72 hours. The model produced another 2x improvement on the throughput

— Sachin Katti, NVIDIA fireside chat
Key Insight
The 3x came from hardware, but the 2x came from the model doing kernel-optimization work that used to take engineers weeks. Katti's real claim is a shrinking loop: better chips let the model optimize itself for the next generation of chips.
06Forward Look

Heterogeneity Is a Fact of Life

GPUs, CPUs, networking, and storage all have to be stitched together, and Katti says a small team like OpenAI will increasingly hand that orchestration to its own agents instead of hand-tuning it.

we will build with our partners like NVIDIA, the right tooling, the right telemetry, expose the right knobs, and really let agents go at it to figure it out

— Sachin Katti, NVIDIA fireside chat
Key Insight
The tell is expose the right knobs. Katti is not replacing the orchestration software with a model; he is building a controllable surface the model can act on, the same pattern as the Astra loop applied to the whole data center.
07Counterintuitive Take

Alignment Is a Reason to Buy More Compute

Asked whether pause talk slows the build-out, Katti argues the opposite: making frontier models safe means training extra safety and alignment models, which needs more compute, not less.

if anything, we believe that we will need more compute to make sure that future models are more safe and more aligned

— Sachin Katti, NVIDIA fireside chat
Key Insight
It is worth noting that the speakers both sell and buy compute, so the conclusion is self-serving. But the mechanism stands on its own: if alignment is done with learned models rather than hand-written rules, safety becomes a compute cost that grows with capability.
08How It Actually Lands

The 15-Minute Rule for an AI Pre-Check

Before the required human review of CUDA-driver code, NVIDIA engineers can run an agent pre-check for known bugs, and Buck found developers accept a fifteen-minute check but complain about anything slower.

15 minutes seems to be the good. Developers are good with a 15 minute tool on a pre-check. If you go any longer than that, they start to complain just regardless of how useful it is

— Ian Buck, NVIDIA fireside chat
Key Insight
The lesson is about adoption, not capability. The first version ran one to two hours and developers hated it; the pre-check only stuck once GPT's token efficiency made it fast enough to tolerate.