Bloomberg Technology

Rodrigo Liang on why inference broke everything open

Rodrigo Liang· CEO of SambaNova Systems at SambaNova Systems
·~7 min·English·Bloomberg
InferenceAI InfrastructureAI CompanyBusiness Strategy
TL;DR

SambaNova's CEO argues that inference has broken the AI market open, and that a challenger can win it by out-flanking NVIDIA on decode throughput, power per rack, mature-HBM supply, and on-prem enterprise data rather than bidding for the newest memory.

01Core Thesis

Inference broke everything open

SambaNova stakes its whole focus on Liang's claim that the inference market has broken everything open, and its $1B Series F is built to turn that surging demand into racks delivered to customers.

the inference market has broken everything open, and that's where we focus. And so we're seeing this incredible demand in the market for inferencing and inferencing at really high token speeds.

— Rodrigo Liang, Bloomberg Technology
Key Insight
The tell is the phrase “broken everything open.” Liang isn't pitching a better GPU — he's arguing the market itself moved. When the bottleneck shifts from training a model once to serving it on every request, whoever is purpose-built for that serving step gets a fresh opening, and a $1B round is the bet that the opening is real.
02The Technical Wedge

Split the inference, not the bill

Inference splits into two phases, and at scale SambaNova pairs NVIDIA for prefill with its own racks for decode to claim two-to-three-times the throughput on the same infrastructure and the same cost.

you're getting two to three x throughput advantage on the same infrastructure and the same cost.

— Rodrigo Liang, Bloomberg Technology
Key Insight
The clever part is that SambaNova doesn't ask you to rip out NVIDIA. It slots in only at the decode step — the memory-bound half of inference where GPUs are least efficient — so the 2-3x is won on the same power and the same budget. Liang's own chip is framed as 5-10x faster on that decode phase; the system-level 2-3x is what survives once prefill still runs on NVIDIA.
03The Power Constraint

Power is the budget

SambaNova's data-flow racks are pitched at roughly 10 kilowatts each against about 100 for a typical GPU rack, so in a power-capped data center the real contest becomes output per watt, not chips per rack.

you can actually run these models at a really, really high performance, very low power, 10 kilowatts per rack versus a 100 kilowatts on a typical GPU rack.

— Rodrigo Liang, Bloomberg Technology
Key Insight
In a data center you can't buy unlimited power, so the unit of competition isn't chips-per-rack, it's output-per-watt. At 10kW versus 100kW, the same power envelope holds roughly ten SambaNova racks or one GPU rack — which quietly reframes “can you get the chips” into “can you afford the electricity,” the wall hyperscalers hit first.
04Business Model

Sell the racks, not the tokens

SambaNova deliberately sells racks to service providers, clouds, and model builders rather than tokens to end users, staying one layer upstream — with its customer count climbing from dozens today toward triple digits by year-end.

we're the business of moving racks. Our customers are service providers, cloud players, your model builders.

— Rodrigo Liang, Bloomberg Technology
Key Insight
Selling racks instead of tokens is a deliberate choice to stay one layer upstream — the picks-and-shovels layer. It trades the fat margin on tokens for a position every token-seller depends on, which is why the customer count (dozens now, triple digits by year-end) is the number to watch, not any single token price.
05Market Expansion

The enterprise is waking up

While attention stays fixed on frontier labs and hyperscalers, the enterprise segment of inference is waking up — and JPMorgan picked SambaNova to run private, regulated inference inside its own firewalls.

one big segment of inference or AI is waking up, and that's enterprise.

— Rodrigo Liang, Bloomberg Technology
Key Insight
The frontier-lab spotlight hides where a durable moat may sit. On-prem, inside-the-firewall inference is exactly what a regulated bank like JPMorgan can't hand to a shared cloud — so the enterprise segment isn't merely late, it's structurally reserved for whoever can put the racks on the customer's own floor.
06Supply-Chain Strategy

Win with the memory nobody's fighting over

Rather than out-bidding NVIDIA for the newest HBM, SambaNova uses mature previous-generation (N-1) HBM already in volume production to generate far more supply — while still keeping enough HBM to run trillion-parameter models without quantizing.

We used HBM that was n minus one technology, HBM that was already in mature production, which allows us to actually generate significantly more supply

— Rodrigo Liang, Bloomberg Technology
Key Insight
This is supply-chain judo: instead of out-bidding NVIDIA for the newest HBM, SambaNova wins by using the memory nobody's fighting over. And it still refuses to go SRAM-only, keeping HBM so it can run trillion-parameter models at full size — betting that “run the hard models on abundant older memory” beats “run smaller, quantized models on scarce new memory.”