LI Luca Ingianni Embedded software consultant working on agile methods, requirements engineering and continuous integration for embedded systems teams.

“LLMs love creating mocks and mocking all of the things, and it just is not going to be very helpful. So I advocate for keeping LLMs on a very tight leash if you are using them to write tests, as you should. … Whenever I have an LLM generate test suites for me, I will always find out like 90% of the test cases are fantastic and do exactly what I want. And the other 10% are just garbage. You must never let go of the reins when it comes to LLMs.”

The Agile Embedded Podcast · Hands-On Engineering Podcasts · November 2026

“LLMs love creating mocks and mocking all of the things, and it just is not going to be very helpful. So I advocate for keeping LLMs on a very tight leash if you are using them to write tests, as you should. … Whenever I have an LLM generate test suites for me, I will always find out like 90% of the test cases are fantastic and do exactly what I want. And the other 10% are just garbage. You must never let go of the reins when it comes to LLMs.” — Luca Ingianni, The Agile Embedded Podcast

Context

Jeff had argued that LLMs make test infrastructure cheap to build. Luca agrees they lower the activation energy but warns about what they produce when asked to write test suites.

Transcript

The Agile Embedded Podcast Around 33:52 into the episode
Luca Ingianni

…that was fully intended to be simple and sort of cutting a couple of corners and not actually being production ready, it just had to be Luka ready and sort of give reasonable benchmarking results. It is a lot of work. So be careful what you're setting out to do because it has a habit of ballooning.

Jeff Gable

Fair enough.

Luca Ingianni

And by the way, case in point, to get back to the topic at hand, the same will happen if you use an LLM to write test suites for you because they love writing tests. They will write all of the tests. And in fact, they will write all of the mocks. I hate that so much. LLMs love creating mocks and mocking all of the things and it just is not going to be very helpful. So I advocate for keeping LLMs on a very tight leash if you are using them to write tests as you should. I absolutely say you should use them because it lowers, just like you said, Jeff, that activation energy. It makes it much more inviting to actually write tests, but you still need to be in control of the entire thing. You still need to consider test cases and ask, well, is this a worthy test case? And then is it actually implemented in such a way that it will remove the risk that I'm worried about? Because whenever I have an LLM generate test suites for me, I will always find out like 90% of the test cases are fantastic and do exactly what I want. And the other 10% are just garbage. So you must learn. I'm sorry, I keep going on about this. Because that has been the thing that has been most on my mind for the last couple of months. You must never let go of the…

Jeff Gable

way. Yeah. All right. I think we're running long on time, so that's probably a good place to cut it. Yes. I like it. Yeah. Why testing sucks, revisited?

Luca Ingianni

Yes. Well, does it still suck? Or did it suck to begin with? I don't know. What's your take?

Jeff Gable

Oh, I don't know. That's just a clickbaity title. Let's be honest here. Right. So, anyway, I hope that discussion was useful and valuable for you and maybe made you think about some new perspectives. Luca, where can people go to find you online?

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from The Agile Embedded Podcast