Things Cedric Clyburn Has Said on Podcasts

“They estimated, I think, six months ago, that your $200 subscription, like the Claude Max, would give you, as a personal user, about $20,000 of API tokens. … More recently, it's about $14,000. So you're kind of starting to see the amount of tokens you get for a specific plan go down. … Yes, these are definitely subsidized, but I think the subsidization is going down.” — Cedric Clyburn, Kubernetes Bytes

Asked whether AI companies subsidize users, Clyburn cites a third-party industry estimate, from memory, of what a flat-rate coding subscription is worth at API prices, and says the subsidy is shrinking.

Kubernetes Bytes · 2026-11-09 Permalink → Listen →

Kubernetes Bytes Around 06:19 into the episode

Cedric Clyburn

…of weights and KV cash. It's going to be a lot of things. But the subsidization of inference is a really interesting conversation, both for developers, but also from an economic standpoint. There's a really good source that I read up on called Semi-Analysis. Have you heard of it before?

Bhavin Shah

Nope, not me.

Cedric Clyburn

It's a very nerdy type of publication where they talk everything about inference and everything about model cost and tiers. And they estimated, I think, six months ago, that your $200 subscription, like the Claude Max, would give you, as a personal user, about $20,000 of API tokens. Yeah. So you could say they're subsidizing. Yeah, they definitely are. More recently, it's about to be November 2026. It's about $14,000. So you're kind of starting to see the amount of tokens you get for a specific plan go down. But remember, this is the API cost that they're charging external customers, right? Like anybody off the street. You have companies like Cursor who are reselling…

Bhavin Shah

Yeah. You know, carrot is string. Yeah. The carrot is a stick. Right now, yeah, eventually we'll get this thing at some point in time. Once they have, I don't know, Anthropic goes public and they have their first quarterly earnings report and the stock tanks are like, okay, we need to bump up the…

Cedric Clyburn

Yeah, can't wait to see that. Well, I don't hope to see that, but I can't wait to see that too. True. So it's an interesting time.

Bhavin Shah

Yeah, for sure. Okay. So now, let's say I've gone through this economic analysis and figured out that I want to run my own models. What are some of the SLOs that I need to think about before going and deploying something on Kubernetes on day one, right? Like, how do I plan for this? Can you talk…

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

“For a standard agentic workload, one prompt that you would give, it could generate 20 gigabytes of space … on a GPU, which is very scarce. … That's why everyone wants their hands on GPUs, because you're not just loading the model weights, but you're generating KV cache as you send requests to the model.” — Cedric Clyburn, Kubernetes Bytes

Asked what model weights and the KV cache are, Clyburn explains that serving a request builds a large short-term cache in GPU memory on top of the static weights, which is why GPU memory is so scarce.

Kubernetes Bytes · 2026-11-09 Permalink → Listen →

Kubernetes Bytes Around 17:41 into the episode

Cedric Clyburn

Yes, I spent a lot of my time on Hacker News and Reddit. The local Lama, so local and then LLAMA, is my favorite source of information for if you want to run your own models, see what people are doing with models. And it's just really cool, good vibe over there, I recommend. We'll

Bhavin Shah

…we include these links as when we share those on the episode in our show notes as well. You already spoke about KVCache, right? So let's talk about, like, let's peel the onion and talk about the technical aspects as well. So what are model weights and what's the KV cache and what does it stand for?

Cedric Clyburn

…for all that input and generates what will be the output. But this takes a lot of space because of how many layers it has to go through in a model. So the model has these different layers in the transformer architecture that it keeps on going through to figure out what the response will be. And each of that is a matrix multiplication, which then adds up to so much data. And I'm talking about for a standard Agentic workload, one prompt that you would give, it could generate 20 gigabytes of space in order to generate that. And this is space of memory on a GPU, which is very scarce. So the inference engine handles this, but no matter what you use, you're generating that KV cache. And so if you ask the same question again and you use some maybe the same tools and look at the same file, that's already stored. You don't have to recompute that. So you'll get faster time-to-first token…

Bhavin Shah

Okay. So the KV cache, like, is it a shared resource between different users, or is it like, I don't know, each user gets their own cache and it maintains some sort of history or cache for the conversation that I'm having.

Cedric Clyburn

Yeah, yeah, yeah. That's a good question because this is something that's being worked on in the space of gateways. Because by default, with naive, you know, loading a model into a GPU and running inference, it's all going the same space.

Bhavin Shah

Okay.

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

Things Cedric Says on Podcasts

Where to Find Them

What They Said

Collections They Appear In