DA Diogo Almeida On The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

“As far as I can tell, the field over the last nine months has learned that benchmarks don't mean shit. Anthropic models are just the best, even if the Chinese models match them on benchmarks. Even when you measure benchmark maxing, you still think the Anthropic models are better than the competition, even though they're not on the Pareto curve of pretty much any benchmark.”

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch · Startups & Venture · November 2026

“As far as I can tell, the field over the last nine months has learned that benchmarks don't mean shit. Anthropic models are just the best, even if the Chinese models match them on benchmarks. Even when you measure benchmark maxing, you still think the Anthropic models are better than the competition, even though they're not on the Pareto curve of pretty much any benchmark.” — Diogo Almeida, The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

Context

Stebbings asked why the big labs can't copy what Almeida's company does. Almeida argues the field has stopped trusting benchmarks as a measure of model quality. It lands in a week when Alibaba claimed Qwen4-Max beats Western models on coding benchmarks.

Transcript

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch Around 19:26 into the episode
Diogo Almeida

fucking clue how little they care.

Harry Stebbings

What? How? They want enterprise. They want revenue. How are you going to get a CIO to sign up if you don't have reliability?

Diogo Almeida

I think people have a hard time understanding intelligence as a product, which is why people try to use benchmarks as band-aids. You know, like benchmarks are kind of a proxy for understanding the intelligence of the underlying thing. But as far as I can tell, the string field, you know, over maybe like the last nine months have learned that benchmarks don't mean shit. Anthropic models are just the best, even if the Chinese models match it in benchmarks. And like even when you like measure benchmark maxing, you still think like the anthropic models are better than this, better than the competition, even though they're not on the Pareto curve of pretty much any benchmark. Basically, it looks like the subfield of like system one or decision models learned this in like, I think there was like competition maybe like starting a week after. It felt like they people learned it in like two, three weeks, where we're like, oh yeah, the other stuff is just benchmark max. It's total crap. It's very easy to see that it's different. I think that it takes a different type of organization to again really, really care about reliability.

Harry Stebbings

If they don't care about reliability, what do they care about then, Diogo?

Diogo Almeida

Well, it really depends on like their business objectives, right? Back in the chat GPT is everything days, this is before Claude Code, they cared about like monthly active users. That's actually a very different metric to optimize for than the best chatbot experience. This is how psychofancy…

Harry Stebbings

When you look at the decision model product landscape for models that you're in, what does that look like in three years? Is it like an Uber Lyft market? Is it like a GCP, AWS, Azure market? Like what is the market composition? Look like in three years' time.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch