“LLMs, they're non-deterministic. I get nervous about what they're going to get right or wrong. We very frequently talk about putting a human in the loop. But if your parser is right 95% of the time, a human is unlikely to notice the 5% where it was wrong, because they're just so used to it being correct. They're just constantly hitting the yes button. This is just human nature. Alert fatigue.”
sputnikintelligence.com
Minsky asked how language models change the job of turning clients' chat messages into trade requests at Jane Street. Casale says they can handle messy, typo-filled messages, but a person reviewing an almost-always-right parser soon stops really checking it.
…thinking about like the extent to I start thinking about our clients is thinking about like, oh, what kind of questions do my sales traders want to be asking in order to figure out how to support like their clients and then building systems that can kind of facilitate that question and answering.
Okay. So now we're in this new like large language model world. Like how does that change the parsing story?
It's kind of opened the doors to being able to parse things that are maybe coming through in a less structured way. So it's like counterparties who are actually like typing things in by hand every single time and maybe are doing different formats. Maybe they have like very cons typos. You can imagine we can start handling those things. At the same time, LLMs, they're non-deterministic. I think that I get nervous about what things that they're going to get right or wrong. And I think it's kind of hard to tell like how comfortable we should be even inserting them into a process. And a thing that I think about very frequently is like we very frequently talk about like, okay, well, we can put a human in a loop and then like a human can give us some sort of like fact checking or some sort of saying like, yes, this chat message looks like this parse. But when you're kind of thinking about when, when does that work? Like if your parser is right, I don't know, 95% of a time, a human is kind of unlikely to notice that 5% of a time in which it was wrong because they're just so used to it being correct. They're just constantly kind of hitting the yes button. This is just like human…
I mean, you could literally insert errors, not like make the LLM make a mistake, but like insert errors into the confirms at some rate and see whether people catch it. And when they don't, you're like, actually, that was wrong. Yeah.
So this has crossed my mind.
I'm not sure how much the sales traders would enjoy this process. Yeah, I