Conrad, who runs a GPU compute marketplace, is describing what a healthy price curve for compute should look like. His point is that the rental price of top-end Nvidia chips sits far below what the same chip can earn selling inference, a gap he says can't last.
Like, I think this will make intuitive sense. It's basically like, if I have milk and it is about to expire, I will lower the price on it. Yes. And so that's basically what we see. So it's the
NVIDIA release cycle, basically, right?
Well, we're more saying if there's a block of compute and it's approaching now, it can't be sold in the past. And so it has to drop its price in order to clear at some price. And so initially, you see this drop where stuff that's sold right now, anything that's on the market right now, is something that hasn't been sold. And so therefore, the price will drop. And then it goes up to where it wants to be. And then maybe compute in the future is worth less because people don't want, like, people would like the compute now. And so they're sort of paying for the shorter lead time. And they pay less for the compute that's off into the future. This is what I think a normal healthy curve looks like. The place where I'm like, oh man, no shit. Is that right now, if you took a B300 and you tried to run inference on it, you can make $40 an hour or so selling that any model, basically. And so that's if you use it at 100%. And maybe if you only have like 50% utilization or something, so like $25 an hour. So $25 an hour. Meanwhile, you can buy B300s at like $5 to $6 an hour. One of these numbers. Has to change. Yeah, it's a huge arbitrage.
It's a huge change. I've mentioned this in your talks a lot.
Yeah, so a bunch of people are basically looking at this and thinking: if I think prices are going to go up, I might be willing to pay more than what I previously thought in the future because I might think, okay, I can buy a five right now, but if I can't buy anymore, then maybe I will be willing…
Cool. So is it, okay, and just I just really want to make super clear on this. Of course. Do you care about the different generations of chips, or do you just care about overall flops and you spread it among whatever chips you want?