Bloomberg Technology

Chey Tae-won on why AI turns memory demand exponential

Chey Tae-won· Chairman of SK Group at SK Group
·~30 min·English·Bloomberg
AI InfrastructureInferenceBusiness StrategyGPU
TL;DR

SK Group's chairman argues the AI era rewired memory demand — from a boom-bust commodity indexed to device counts into the structural bottleneck of inference, where every agent's KV cache has to live somewhere.

01Core Mental Model

From Per-Person to Per-Agent

In the old world memory demand was capped by the number of people and devices, so it ran in booms and busts; in the AI era each person may run tens or hundreds of agents, and the curve turns exponential.

you might had that the tens over the hundreds different of your own agent so that every agent need that a lot of memories

— Chey Tae-won, Bloomberg Technology
Key Insight
Chey is quietly changing the denominator of the whole memory business. For decades demand was a function of how many humans bought how many devices, which is why it boomed and busted. Pin demand to agents-per-person instead and the ceiling disappears — because software, not population, now sets how many memory-hungry things exist.
02The Mechanism

Inference Makes Memory the Bottleneck

The reason demand explodes is the KV cache: every agent generating tokens produces one that has to be stored somewhere, which makes memory the real bottleneck in the inference market.

because that the memory is a kind of, uh, real bottleneck and the we have to solve it.

— Chey Tae-won, Bloomberg Technology
Key Insight
A memory-maker just narrated the roofline model from the other side of the fence. GPU people describe inference as memory-bound because the KV cache saturates bandwidth; Chey describes the same physics as a demand signal for his product. The interesting move is that he treats the KV cache as the load-bearing reason memory is no longer a commodity.
03The Hierarchy

Hot DRAM, Cold Flash

Because you cannot fit every agent's context in the fastest memory, the KV-cache era forces a pyramid: DRAM and HBM hold the hot, instant data while NAND flash stores the cold context you retrieve on demand.

that's the kind of, uh, the pyramid, the hierarchy, uh, of the what? The whole memory storage systems.

— Chey Tae-won, Bloomberg Technology
Key Insight
Chey is describing, from a chairman's chair, the exact tiering problem that serving engineers solve with paged KV caches and cache offload. The business consequence is that SK does not want to sell one product; it wants to sell the whole pyramid, because the inference workload needs every tier and the hard part is moving data between them fast enough.
04The Demand Signal

"That's Not Enough"

SK Hynix plans to double total capacity within five years and analysts warn of oversupply, but Chey says his customers call even a doubling insufficient and ask for five or six times more.

But my customers said that that's not enough. We need that more.

— Chey Tae-won, Bloomberg Technology
Key Insight
The real innovation here is not the fab, it is the long-term agreement. By locking both volume and price with customers who are desperate for supply, Chey converts a notoriously cyclical commodity into something closer to an annuity — which is exactly what lets him justify the CapEx that analysts read as reckless oversupply.
05The Origin Bet

The Business Nobody Wanted

When Chey acquired Hynix about fifteen years ago it had spent over a decade in a bank workout, burdened by cyclical multi-billion-dollar CapEx, and his bet was that memory is the most sophisticated manufacturing business there is — so if you can really do it, you can do anything.

this memory, uh, manufacturing is most sophisticated, uh, the manufacturing business. So if you Really do it, then you can do anything.

— Chey Tae-won, Bloomberg Technology
Key Insight
Notice what Chey actually bet on. Not that AI would arrive, not that demand would spike — those were unknowable fifteen years ago. He bet that owning the most sophisticated manufacturing discipline there is would eventually pay off no matter which demand showed up. AI is simply the payoff event; the option was mastery.
06The Economics

The Metric Is Token Cost

Chey reframes the AI-bubble debate as a question about the cost of intelligence, and points out the cost of a token has already fallen roughly five to tenfold in three years — with memory now one of the levers pushing it down.

within a three years, actually, token cost is about one fifth of what, the one tenth.

— Chey Tae-won, Bloomberg Technology
Key Insight
When a memory CEO frames his contribution as lowering token cost, the yardstick has quietly changed. HBM has stopped competing on gigabytes and started competing on cost-per-inference — the same economic metric as GPUs and networks. That alignment is why Chey can wave off the bubble talk: to him the stock price overshoots, but dollars-per-token is a real, falling number.
07The Forward Bet

Memory as a Service

Chey floats a way out of the commodity trap: instead of shipping raw chips priced by the gigabyte, sell tuned, managed memory systems plus software for the inference market — which he admits is still just an idea.

we could actually memory services that the memory as a service.

— Chey Tae-won, Bloomberg Technology
Key Insight
This is the same escape route NVIDIA took with networking: stop selling a part and start selling a co-designed system with software wrapped around it. If memory becomes a managed service tuned for the inference market, SK stops being a price-taker on a boom-bust commodity and starts capturing margin on the hardest problem in serving — keeping the KV cache fed.