Andrew Dai on the Jagged Frontier That Left Vision Behind
Andrew Dai argues AI's progress is a jagged frontier — really strong at math, coding and Go, still poor at counting the glasses on a table — and he raised $55 million to post-train that gap.
The Frontier Is Jagged, Not Even
AI progress is uneven — really strong at math, new physics ideas, coding and Go, and still beaten by a child at reading a photograph.
I think what a lot of people don't realize is that the way AI is progressing is a jagged frontier.
Off By One and You're Mad at Your Fridge
Vision models handle the gist of a photo well, but the uses Dai describes turn on exact answers — the count, the connection, the row — and being off by one or two is enough to make you mad at your fridge.
But imagine if you if these models really understood exactly what you had in the fridge, especially the count, right? If you're off by one or two, you would be really mad at your fridge.
Thirty Pixels Can't Hold a Fridge
The multimodal benchmark that shows visual reasoning improving feeds models 32-by-32 pixel images, a canvas too small to hold the scenes any real use case cares about.
But, if you look at the details, if you really uh try to understand what's going on, that benchmark uh the inputs are just 32 by 32 pixel images. Although, the newest one is uh like 64 by 64 pixel images.
Raise the Number the Math Produces
Post-training is cheaper than pretraining but far more than a few million dollars, and Dai says their compute and data math put the amount they needed at the 50 million dollar mark.
Our goal was to raise the amount of money we believe that we needed to get things done and that was the 50 million dollar mark from our math. That would give us enough compute and data budget to build a world-class model.
They Turned Down the Higher Valuation
Offered higher valuations with more capital, Dai took less to get investors who understand that an AI company may be revenue free for a while and won't flinch at how he spends.
So, we did have offers for higher valuations actually. Higher valuations with more capital, but for us it was really important to have the the right people on our cap table, the right investors who can who understand this journey of an AI company
The Clock Belongs to the Market, Not the Cap Table
The pressure Dai answers to comes from the market rather than his investors — new models land every two weeks at some companies, and months spent fundraising are months the frontier labs move without you.
definitely there's a ton of pressure uh on all sides, but I think I would say the majority of that pressure isn't from investors, it's from the market, the AI industry, right?
Don't Pitch the KV Cache
Pitching MQA, MHA or KV caches to investors fails twice over — they may not follow, and the method may be irrelevant in months — so Dai frames the big picture and lets the technique sit underneath it.
Like if you're talking to an investor about like MQA, MHA, uh or like KV caches, then um there's quite a good chance they won't know what you're talking about.
The Recipe Came From Experiments, Not a Grand Vision
The 2015 paper Dai is asked about wasn't the execution of a grand vision but the residue of many experiments, which is why his advice is to tinker first and raise only once the results surprise you.
the original paper that you mentioned from 2015, that paper wasn't designed. It wasn't uh the case that we had this grand vision and then we implemented it and then it worked.