Latent Space

Anima Anandkumar on One Model for All of Physics

Anima Anandkumar & Benedikt Jenik· Co-founders at Accelerated Understanding
·~27 min·English·Latent Space
MultimodalTrainingAI InfrastructureAI Company
TL;DR

Anima Anandkumar and co-founder Benedikt Jenik are building Accelerated Understanding, a single foundation model for the physical world: many domains of physics in one model, trained on data from numerical simulators and improved by the laws of physics themselves, using neural operators to reach context lengths of trillions where transformers cannot.

01The Bet

Everything Into One Model

Language models won by pushing every task into a single general model instead of many narrow ones, and Accelerated Understanding is making the same bet for physical simulation.

let's just put everything into one model and that turned out to work phenomenally well and exceed pretty much everything else those specialized models. And so now we were asking ourselves, can we do the same thing for physical simulation and for physical understanding?

— Benedikt Jenik, Latent Space
Key Insight
The wager is not that physics resembles language, but that the training recipe transfers: consolidation already beat specialization once, so if physical domains share enough structure, one model should again beat a shelf of narrow surrogates.
02The Worldview

You Cannot Download New Physics

Scientific discovery is by definition not in any dataset, so a purely data-driven model tops out at the average of its data, and the laws of physics become both the extra teacher and the way past that ceiling.

if you think about scientific discovery, it's about doing something new. So, by definition, it cannot be there in the training data. And so, this reliance on just purely datadriven AI is not going to be enough. And that's where you know adding the laws of physics is really critical.

— Anima Anandkumar, Latent Space
Key Insight
Physics plays the role human feedback plays for language: a signal absent from the raw data. But where a thumbs-up is sparse, a physics-law residual is dense and computable, so the model can grade and improve itself without waiting on a human.
03Why It Transfers

Same Equations, Different Regimes

Fluid in a catheter and fluid in a rocket obey the same equations at different settings, and deeper still, physics shares features like conservation, causality, and time, so one model can carry learning across very different domains.

there is something that is shared among them and that's what we've seen this play out that across different domains if you think energy semiconductor aerospace you know it looks very different but there are common underlying principles

— Anima Anandkumar, Latent Space
Key Insight
The claim is stronger than analogy. If disparate equations share invariances like conservation and causality, a model that learns those once can reuse them everywhere, which is precisely why a joint model could beat specialized ones.
04The Key Result

Multi-Physics Beats Specialization

A single model trained on several kinds of physics outperforms separate models of the same size each trained on one, which points to shared, emergent learning rather than just more parameters.

it turns out that having the model of the same size with multiple areas of physics does better than giving all of those parameters to each single physics. So if you had separate models and made them big enough as the original one, it still is worse.

— Anima Anandkumar, Latent Space
Key Insight
This is the load-bearing result. Scaling laws say bigger is better; this says broader is better too, with size held fixed. If it holds up, parameter count alone cannot explain the gain, and the diversity of physics in one model contributes.
05The Hard Wall

Four Dimensions, Trillions of Tokens

Physical data grows across three spatial dimensions plus time, all independently, so a single sample can reach trillions of elements, far past the roughly million-token contexts language models handle.

when you multiply those numbers out, you're very quickly in the billions or even trillions in context. And that is something we've actually achieved with our models. Like we're able to train up to a trillion context input.

— Benedikt Jenik, Latent Space
Key Insight
Context length here is the physics, not a convenience. Video models compress and patch pixels to dodge the dimensionality; engineering-grade simulation cannot discard resolution, so the context explosion is forced rather than chosen.
06The Architecture

Neural Operators, Not Transformers

Transformer attention scales quadratically and cannot reach trillions of context, while neural operators stay resolution-invariant, so the same model can run a cheap low-detail pass or the most demanding physics.

using transformer architectures that have worked so well for language that just wouldn't be able to support a 5 trillion context length no matter all the compute in the world is thrown at it.

— Anima Anandkumar, Latent Space
Key Insight
The efficiency argument is also a physics argument. All-to-all attention assumes any point can depend on any other, but physical fields are structured rather than arbitrary all-to-all, so neural operators are not merely cheaper, they match the shape of the domain.
07The Systems Problem

One Sample Will Not Fit a GPU

The models' layers and even single training samples are too large to fit on one accelerator or a full node, so standard sharding like FSDP breaks and the team rebuilt the sharding strategy from scratch.

our layers are so big you can't reassemble it inside a GPU. Our data samples are so big they don't fit an accelerator or even a full node. So we had to reinvent this whole sharding infrastructure, the whole sharding strategy

— Benedikt Jenik, Latent Space
Key Insight
Here the frontier-lab analogy runs in reverse. Large-model infrastructure is only now walking into problems, like samples that do not fit a node, that physics AI faced from day one, so the HPC-plus-modern-tricks toolkit is a head start, not catch-up.
08Where It Lands

Physics Moves Into the Design Loop

In chip design, today's pipeline freezes the digital design and runs physics as a one-time check, leaving performance headroom that a physics model kept in the loop could use.

we're not taking advantage today of that headroom to get extra performance. So there's this potential to unlock much more of the skill set when you're also able to grab into what does the physics allow me to do to push the envelope a bit more

— Benedikt Jenik, Latent Space
Key Insight
Notice the wedge: not replacing the chip-design stack, but harvesting the safety margin it already leaves on the table. That makes the first sale a performance upgrade on an existing flow rather than a rip-and-replace, which is far easier to buy.