TS Tuhin Srivastava Tuhin Srivastava On The Generalist

“Inference is one of the most misunderstood markets in the world, because people think you just take vLLM, slap it onto GPUs and call it a day. In a supply-constrained market, operating at large scale with what I would call quite flaky technology, GPUs fail all the time, chips fail all the time, and running at very high utilization in a market where demand is growing exponentially, there's so much complexity at every layer.”

The Generalist · Startups & Venture · November 2026

“Inference is one of the most misunderstood markets in the world, because people think you just take vLLM, slap it onto GPUs and call it a day. In a supply-constrained market, operating at large scale with what I would call quite flaky technology, GPUs fail all the time, chips fail all the time, and running at very high utilization in a market where demand is growing exponentially, there's so much complexity at every layer.” — Tuhin Srivastava, The Generalist

Context

Asked how Baseten's financing needs change as it buys more compute, Tuhin first explains why inference is much harder than people assume. He then describes typical terms for large compute deals.

Transcript

The Generalist Around 32:16 into the episode
Tuhin Srivastava

Just because the supply chain is so scaled up.

Mario Gabriele

As you start to continue to have to scale up that supply, how do the different financial requirements change for your business? Do you start to have to build up a big debt practice? Or, you know, I could imagine lots of different ways of getting the oxygen into the system that you need.

Tuhin Srivastava

Oh, totally. I don't know about live debt practice, but one of the very interesting things about inference is like inference in a lot of ways, I think it's maybe like one of the most misunderstood markets in the world because people think you just take VLM, which is open source framework, you slap it onto GPUs and you call it a day in a supply-constrained market, operating at large scale with what I would call quite flaky technology, which is, you know, GPUs fail all the time, chips fail all the time, and running at very high utilization in a market where the demand is growing exponentially. There's so much complexity at every layer of this. And then when you add the fact that there is CapEx involved, and whether that means, you know, through building data centers yourself or whether that means through, you know, like if you are looking at a couple of billion dollars of compute expenditure over a given period, in today's market, that probably requires about a five-year term and a 30% prepayment. I guess, you know, play that out. Yeah. So let's say you wanted to do $10 billion of spend. For five years, it's called five billion with 30% prepay. That's one and a half billion dollars of prepay you have to do.…

Mario Gabriele

This is where the two years of Macquarie doing infrastructure

Tuhin Srivastava

ended up being really

Speaker 4

helpful in the end, right? That was probably painful two years, but uh, it's come back around in a way, right? Yeah,

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Generalist