“The stateful nature of LLM inference is fake, in that they are really stateless. The way they do inference is that we send them the entire history of a conversation every single time. … Every conversation really is a very long prompt to the continuation of that conversation.”

…discovered if you wrote that prompt as a conversation, as a dialogue with, you know, with quotes, user says, da, da, da, computer says, the LLM will continue the conversation. And that's how ChatGPT basically was born. Like people noticed this and noticed this pattern. Now, LLMs are being trained with this pattern, but that's how basically originally it worked. So if you think of it this way, every conversation really is a very long prompt to the continuation of that conversation. The way LLM works, you send the history of the conversation and then they continue with the next answer. And then you get back the answer and then you continue with your answer and so on, so forth, so on so forth. If you are using LLMs, you will know that pricing for these things, depending if you're using token-based spend or a subscription, well, if you're using a subscription, that doesn't…