Bloomberg Tech

Aidan Gomez on AI Safety Cartels

Aidan Gomez· Co-founder and CEO of Cohere at Cohere
·~7 min·English·Bloomberg
AI SafetyPolicyOpen SourceAI Company
TL;DR

Cohere CEO Aidan Gomez backs a higher safety bar and public incident reporting, but warns that letting a few frontier labs set and grade the industry's safety rules would build an AI cartel instead of real protection.

01The Core Objection

A Cartel by Another Name

Letting the biggest labs coordinate on safety standards would hand OpenAI and Anthropic the power to set the rules for everyone, which Gomez calls a cartel by another name.

The companies that are building AI absolutely should have a seat at the table, but they should not get to decide for everyone by themselves.

— Aidan Gomez, Bloomberg Tech
Key Insight
The move reframes an antitrust problem as a safety problem: the same coordination that would be illegal for competitors becomes acceptable once it is labeled safety cooperation.
02What He Actually Wants

Raise the Bar, Not the Gatekeepers

Gomez wants a higher safety bar backed by rigorous testing and accountability, but insists the rules come from a broad set of stakeholders rather than a few incumbents granted antitrust exemptions.

we do need a higher bar for safety, and that needs to be backed by rigorous testing, independent scrutiny, and accountability. But we need to make sure that doesn't come from just a few select players.

— Aidan Gomez, Bloomberg Tech
Key Insight
Gomez is separating two questions the labs' proposal blurs together - how strict the rules are, and who has the authority to write them - and only objecting to the second.
03The Alternative

Four Things He's Calling For

His alternative is four concrete asks: an evidence-based risk framework, mandatory incident transparency, evidence-scoped testing, and real assurance standards that a small elite does not grade alone.

the four things that I've called for are, like, an evidence based risk framework. We need to agree on what the risks are, what the harms potentially are, and measure those capabilities.

— Aidan Gomez, Bloomberg Tech
Key Insight
Each ask is designed to move authority outward: frameworks agreed in public, incidents disclosed publicly, and standards no single group can quietly grade in its own favor.
04On Existential Risk

X-Risk Is Science Fiction (For Now)

Gomez thinks the existential-risk debate veers too far into science fiction, and that Terminator-style scenarios should not dominate the public conversation at this stage.

Personally, I think the existential risk debate veers too far into science fiction and is informed by a lot of those stories and a lot of extrapolation.

— Aidan Gomez, Bloomberg Tech
Key Insight
By naming the Hugging Face incident as his example, Gomez is arguing that the loudest safety evidence so far is a staged demo, not a spontaneous loss of control.
05Anatomy of a Scare

The 'AI Escaped' Story, Taken Apart

The headline that an AI hacked its way out described an agent placed in an insecure sandbox and explicitly told to hack, so the escape was arranged by people, not emergent menace.

they put an agent inside an insecure sandbox. They set it on a task of hacking. That was a task that it was asked to perform, and it hacked its way out.

— Aidan Gomez, Bloomberg Tech
Key Insight
The scare works precisely because the setup is invisible in the headline: strip the human-arranged sandbox and the assigned task, and 'the AI escaped' has almost nothing left to describe.
06How to Test Safely

Test the Danger, In a Sealed Room

He fully supports testing high-risk capabilities like cyber, but only inside secured sandboxes with no connection to the open internet.

Those tests need to happen in secured environments.

— Aidan Gomez, Bloomberg Tech
Key Insight
The disagreement was never whether to probe dangerous capabilities, but under what containment - which turns the safety question from should we test into is the test environment actually sealed.
07The Conflict of Interest

An Auditor Who Shares Your Backers

Anthropic's offer of independent evaluation with employee-level access falls apart when the proposed evaluators share the lab's funders, which is a conflict of interest, not real oversight.

There's a very high overlap and conflict of interest present

— Aidan Gomez, Bloomberg Tech
Key Insight
Independence is structural, not a job title: an evaluator inside the lab and funded by the same backers can be technically rigorous and still institutionally unable to deliver a verdict that threatens its own sponsors.
08The Tell

Arguing Against His Own Book

Tougher rules would actually protect Cohere as an incumbent, and Gomez argues against them anyway on the grounds that the field needs more stakeholders, not fewer.

I'm trying to argue against it mostly not out of self interest because there's an argument to be made to the opposite, but towards the fact that we need more stakeholders around the table here

— Aidan Gomez, Bloomberg Tech
Key Insight
Arguing against your own advantage is a costly signal - it makes the objection harder to dismiss as competitive posturing, which is exactly why Gomez foregrounds that the capture would help Cohere too.