Latent Space

Mark Chen on Why You Can't Cheat the Real World

Mark Chen· Chief Research Officer of OpenAI at OpenAI
·~41 min·English·Latent Space
LLMReasoningTrainingAI Company
TL;DR

OpenAI's Chief Research Officer explains why he bets on the exponential, grades progress only against metrics that can't be faked, and runs research like a trader's book of high-risk bets.

01Core Mental Model

The Exponential Holds

Across LLM history people keep declaring a scaling bottleneck permanent, and a new piece of engineering or research keeps breaking past it.

it's held for, you know, almost 10 orders of magnitude, but there's no reason it should not keep keep holding.

— Mark Chen, Latent Space
Key Insight
The bear case has never been wrong that a bottleneck exists — only that it's permanent. Chen treats "pre-training is dead" as a recurring genre of prediction, not a new finding, which is why he discounts it on sight.
02Getting In Without a PhD

Research Taste Is Just Replication

You don't need a PhD to build research taste — you need to fully re-implement papers you admire until the tricks the authors never wrote down surface on their own.

the best mechanism I've found for developing that is really just replication. So, I think you should take papers that you really look up to and just try to fully replicate it.

— Mark Chen, Latent Space
Key Insight
Chen frames research the way he framed trading: find a metric you can't fool yourself on. Replication works precisely because the training curve either matches the paper or it doesn't — there's nowhere to hide.
03Where RL Works

RL Needs Cold Hard Truth

Reinforcement learning takes off wherever an answer can be proven right or wrong, and stalls wherever two experts would reasonably disagree.

for now, it's just where there's cold hard truth, things like math and computer science, where you can prove it correctly or wrong. Um that's where you kind of see it really taking off.

— Mark Chen, Latent Space
Key Insight
This is why OpenAI treats coding as its proving ground: it's the rare real-world task that is both long-horizon and objectively gradeable — the code runs or it doesn't — giving RL a foothold in messy reality that creative writing never offers.
04The Reasoning Bet

Conviction Against a Working Machine

Reasoning almost didn't happen because pre-training plus post-training already worked, and betting against a working machine takes conviction rather than consensus.

even at a company like OpenAI, you would have people ask naturally, why do something when you have a machine that works?

— Mark Chen, Latent Space
Key Insight
The obstacle was organizational, not technical. Even a frontier lab defaults to exploiting what already works; escaping that gravity took a handful of high-status researchers spending their own credibility to steer the whole company onto an unproven bet.
05Measuring Progress

The Evals Crisis

The familiar gold-standard benchmarks are saturating and the moment a good eval goes public it can be gamed, so measuring real progress has become a permanent arms race.

we really are kind of in an evals crisis, right? Where all the really great uh evals that we we all know like growing up like taking the SAT or those those are all fully saturated.

— Mark Chen, Latent Space
Key Insight
The real move is incentive design, not testing. By making the eval team's job to defeat the model team, OpenAI turns Goodhart's Law — any metric you optimize stops measuring what you wanted — from a silent failure into a controlled, adversarial process.
06Why Models Fumble Easy Tasks

The Jagged Frontier Is a Context Gap

A model can win a math olympiad and still fumble a task a person finds easy, and the gap is usually missing context rather than missing intelligence.

context, just um being able to take a single task, learn lessons from it, and apply them to future tasks, um that capability is something that, you know, a lot of people are in AI working towards right now.

— Mark Chen, Latent Space
Key Insight
This reframes the AGI gap from raw horsepower to memory-in-motion. The human edge isn't being smarter on any single task — it's carrying a lesson from one task into the next, which is exactly the continual-learning primitive still unsolved.
07The Future of Research

Vibe Research: Ideas Up, Execution to the Model

Research is turning into orchestration — humans supply the idea and the taste while models increasingly do the implementation — and the three-year goal is models running research end to end.

you're starting to see a lot of the work become mostly orchestration-focused, right? Like the researchers coming up with the ideas and the model's great enough to do the implementation execution by itself.

— Mark Chen, Latent Space
Key Insight
Notice what is not being automated. Taste is the last human moat precisely because it's the "unhackable" judgment that can't be graded — the same reason RL struggles with subjective fields loops right back here.
08The Meta-Strategy

High-Risk Bets Are the Alpha

OpenAI's edge is taking many high-risk research bets, accepting that plenty won't pan out, and needing only the occasional mega-hit to justify the whole portfolio.

that is a big part of OpenAI's alpha because I think one thing that differentiates us from other labs is we take a lot of high-risk bets. And I think that's what's allowed us to stay at the frontier uh so consistently over time.

— Mark Chen, Latent Space
Key Insight
The failure discipline is the hidden asset. Even losing bets ship value, because a good write-up saves the next team from the same dead end — so the portfolio compounds knowledge even in the years it doesn't compound wins.