“What are things that feel very safe to do with a system that we don't fully understand? And what kinds of things feel not safe? We want to pull some data to sense-check whether any clients that previously traded with us have stopped. This is a read-only dashboard. It's not going to cause any trading decisions to change. We're not going to publish this data to any regulators. All of that makes it a very easy thing to vibe code. But if any of those things were not true, now all of a sudden we really do want someone to start thinking about the code.”
Signals and Threads · Hands-On Engineering Podcasts · November 2026
Context
Minsky asked how she weighs the risk of traders using AI to build apps whose code nobody has read. Casale ties the scrutiny to what the code can touch: a read-only dashboard is fine to vibe code, while anything that drives trades or goes to regulators is not.
Transcript
Yeah. Right.
…of safety trade-offs of having, especially less technical users who maybe don't understand the system that well and maybe aren't really coders at all, use this stuff and have LLMs generating things for them that they actually don't understand and that nobody understands it and no human has read.
Yeah. I think, especially on the sales desk, a thing that we've been thinking about a lot is how do we think about varying our scrutiny? And I think this has always been like a skill that we expect devs to have. It's like some things need very, very, very high scrutiny and you really need to stare at it in order to believe it's working exactly the right way. And maybe some things you can like maybe have a little bit less scrutiny on because maybe this thing is like not super, very important or maybe this thing is only going to be used once for something that maybe doesn't super matter. And I think when we've been thinking a lot about like vibe coding our apps on sales, a thing That we've been thinking about is like, what are things that feel very safe to do with a system that we don't fully understand? And what kinds of things feel not safe to do with a system that we don't fully understand. And so maybe like an example of one is like, we want to pull some data to give ourselves a sense check about whether or not any clients that previously were trading with us have stopped trading with us. So this is like a read-only dashboard. It's not going to cause any trading decisions to change. We're not going to go and publish this data to any regulators. All sorts of things about this make this feel like a very easy thing to vibe code…
But then how do you actually deal with knowing what it can do if no one has read the code?
I've never really considered that someone has like built up a dashboard that was then doing something like very completely unexpectedly, like trying to like send a request off to like a database that like it wasn't prompted to do. I guess I could do this thing.
I mean, if Hugging Face is any example, like there's lots of. I actually do think there are real things to worry about there. In part, even if, even if you don't, if you sort of ignore the like, you know, model with its cyber capabilities, guards turned off, running wild and hacking into things…
More from Signals and Threads
