“You want to extract some information from unstructured data, use an extraction model. Then you want to classify what it finds. Then you use a decision model. And then if you want to reinterpret or summarize that for the user, you then pass that into an LLM. So you can create a chain of these models that work together instead of using the LLM to just do everything that is more prone to failure.”
sputnikintelligence.com
The hosts were discussing Cloudflare's new 'extraction model', which pulls structured data from inputs like a photo of a receipt. Kamran lays out how he expects small specialised models to be chained together instead of asking one LLM to do every step.
…type of models that are going to evolve from like what we've learned about LLMs so far and like where they fall apart, right? Because you can do this with an LLM. You can give it a receipt and you can say, give it back to me, give it back to me in this format of JSON, right? This scheme is because
it supports multimodal input.
Exactly. But you're never going to be guaranteed that it's going to give it back to you exactly the same way every time and in the right format. With an extraction model, that's guaranteed. You will always get the schema back that you expect. Now, whether the probabilities are going to be the same, that's where they share a little bit of similarity with the probability, right? Where it's still making a probabilistic guess. So it's still guessing, but the percentage is baked into the output as something that's still deterministic that you can then take action on. So the idea, yeah, the idea here is that these all kind of work together, right? Like you want to extract some information from unstructured data, use an extraction model. Then you want to classify what it finds. Then you use Jev or like a decision model. And then if you want to reinterpret or summarize. That for the user, you then pass that into an LLM, right? So you can create a chain of these models that work together instead of using the LLM to just do everything that is more prone to failure.
Yeah, this reminds me of solving a similar problem, not exactly as like Type Chat was doing way back. And I'll include a link in the show notes, but this was maybe an experiment for Microsoft to strongly type chats, like because of this problem, right? Where you get kind of unreliable output some…
Yep, that's right. So anyway, I thought that was pretty cool. So that's a new, there's going to be an open weight model. It's not, I don't think that it's released quite yet, but Cloudflare says they are going to be introducing an open weight model for this. So that's the new kit on the block that…
Yeah. And it's available in preview now on the workers AI as at CF Cloudflare Cleric.