Anton Leicht on keeping AI power from concentrating
Carnegie fellow Anton Leicht argues that AI power concentrates by default - inside the labs, in Washington, and between the US and the rest of the world - so the realistic goal is not a grand plan but to keep threading the needle, nudging power back toward balance so we muddle through.
The Trend Line, Not the Level
Today's models are not very dangerous at their current level; what worries Leicht is the trend line and two thresholds we are close to - labs whose internal progress outruns any outside oversight, and models good enough at bio and pharma to be genuinely dangerous.
we're nearing a point where the pace of development inside the labs just breaks away from the pace of democratic oversight and democratic insight into what's happening
A Pause Is a Gift to China
A frontier-only pause would let China keep out-producing the US across robotics, manufacturing and energy while it closes the one gap the US leads on - chips - so Washington is very unlikely to accept it.
if you look at all the domains of strategic competition China is basically eating America's lunch in most of them
The Pause Is an Expectations Game
Whether a pause tanks the market is an expectations game: a lab-led pause sold as reliability could even calm investors, but a policy-shock pause reads as radical and sends the smart money running for the exits.
the pause currently still reads as such a radical policy proposal and such an unprecedented policy intervention that I think any conservative market analyst and a lot of the smart money will think well this is getting very volatile
The Gamble He Won't Take
Leicht is less worried about authoritarian states than about liberal democracy, whose monopoly of violence and role as arbiter of disputes both erode once everyone carries a personal superintelligence.
on that gamble I'm much less willing to roll the dice and I'd much rather figure out how we can sort of integrate these capabilities with some functioning version of the nation state
Control Is More Stable in Beijing
Counterintuitively, China's tiered, state-controlled diffusion is a more stable equilibrium than America's open market, so China may absorb powerful AI more smoothly than a more volatile US.
So if that is a stable equilibrium then I think China can stabilize it around it much more quickly than a much more volatile US
Auditors Running on Good Faith
Independent auditors like METR and Redwood are stuck relying on the labs' good faith, and the fastest fix is executive pressure - an approved-evaluator list the labs must actually let in.
apparently they're just too reliant on the good faith of the AI companies because of the voluntary dynamic
Rich, but Disempowered
Most of the world will get richer in absolute terms yet be strategically disempowered, and Europe's one real card is its chip-tooling chokehold, which it can trade for assured frontier access.
building these systems ourselves is just too expensive it doesn't actually work so the next best thing we can think about is how do we get access to frontier models in a way that is assured and secure
Fine Until It Isn't
Leicht's answer to the doomer-shorts challenge is that catastrophe is not a smooth slide you can bet against - things look great until they suddenly do not - and his best case is simply to muddle through by keeping power from concentrating.
a lot of the ways in which things go badly are just like things go extremely well in the market just all the way until they go really badly. And then the only situation where you cash in is when you're dead