Bloomberg Tech

Rayan Krishnan on why AI outruns our ability to understand it

Rayan Krishnan· Co-founder & CEO at Vals AI
·~5 min·English·Bloomberg
AI SafetyReasoningTrainingPolicy
TL;DR

Independent AI evaluator Rayan Krishnan argues the real danger is less about the models being too capable and more that we are building them faster than we can test and understand them, and that honest measurement only works when the tester is barred from also being the fixer.

01The Frame

Panic Is the Wrong Response

The week's most extreme AI warnings mostly create public anxiety; from inside the testing process, the labs have been forthcoming and coordination looks achievable.

I found reason to be optimistic.

— Rayan Krishnan, Bloomberg Tech
Key Insight
His optimism is not a claim that the models are safe. It is a claim about behavior: the labs are willing to be measured. That is an argument about incentives, not about capabilities.
02The Core Gap

We Build Faster Than We Understand

Money and talent have gone into making models more capable, not into testing them, so our ability to build now runs ahead of our ability to understand what we built.

our ability to create better models now outpaces our ability to actually understand them

— Rayan Krishnan, Bloomberg Tech
Key Insight
Evaluation is a public good that no single lab is paid to fund, which is exactly the vacuum that created financial auditors and product-safety testers in older industries.
03The Measurement

When a Model Builds Its Own Successor

Vals AI's RSI index tracks recursive self-improvement, a model making its next version with no human researcher, and extrapolating the current trend points to models passing human researchers around August 2027.

that describes the model's ability to make the successor version of itself autonomously

— Rayan Krishnan, Bloomberg Tech
Key Insight
The 2027 date is a line drawn through a trend, not a measured event. Its value is as an early-warning tripwire that says roughly when to watch, not a promise of what will happen.
04The Current Ceiling

Great Engineers, No Intuition

Today's public models run experiments and act as capable engineers, but they lack the intuition to invent new experiments or reach fundamental breakthroughs, so they cannot yet do AI research on their own.

the models are actually very good at executing experiments and operating as engineers

— Rayan Krishnan, Bloomberg Tech
Key Insight
Vals AI only tests public models, not unreleased systems, specialized agents, or multi-agent setups. So this measured ceiling likely understates what the labs already have running internally.
05The Independence Problem

The Auditor Can't Be the Fixer

If the group that audits a model is also paid to repair it, you rebuild the conflict of interest that produced Enron, so Vals AI keeps its test sets private and refuses to sell fixes or training data to the labs it tests.

And then you end up in situations like Enron.

— Rayan Krishnan, Bloomberg Tech
Key Insight
The precedent he reaches for, ratings agencies and auditing firms, cuts both ways: those industries are for-profit and have been captured before. Structure, not good intentions, is what keeps an evaluator honest.
06What Happens Next

It Only Works If Everyone Cooperates

A single evaluation standard holds only if every frontier lab agrees to be measured the same way, and Krishnan is betting that rational self-interest gets them there with or without regulation.

I have a lot of reasons to be optimistic. I think that people at these labs are rational actors

— Rayan Krishnan, Bloomberg Tech
Key Insight
This is a coordination game where going it alone is costly. The optimistic case rests on the labs pricing in reputational and systemic risk, which is precisely the risk an independent evaluator makes visible.