“I also have had cases where I'm like, fix this test. And the AI was like, okay, I'll delete the test. And I'm like, well, technically, we don't have broken tests now, but not what I meant.”
The InfoQ Podcast · Hands-On Engineering Podcasts · October 2026
Ebertz is explaining why AI-generated code and AI-maintained tests need to be kept apart. Her team's agent instruction files now spell out rules like "don't just delete tests."
Transcript
that. I don't know that I have great insights, but when I first started playing with AI, one of the first things one of my teammates was like, look, it's great. It generates the code and then it fixes all the tests it broke. And I was like, wait a minute, like just doing whatever it wants.
You could write it, assert true, and then the test passes. Yeah,
exactly. And I also have had cases where I'm like, fix this test. And the AI was like, okay, I'll delete the test. And I'm like, well, technically, we don't have broken tests now, but not what I meant. So I think we do need to figure that out. I mean, I think you can still have AI generate tests, but I think you need to make sure that it's separate or there's clear commands that like as a part of this rewrite, you don't touch any of the tests or, you know, much clearer on that side than we are right now. In general, at least. I know like our code base, we now have in our like agent files things like, you know, don't just delete tests and things like that. I
mean, this is one of the places where you've talked about avoiding perfectionism, but here's almost the place where you have to be perfect or aim for perfectionism, especially on the tests end, because if you're going to rely on the AI to generate the code, all right, there regenerates slop. Okay…
Yeah, it's interesting because I guess like, and this is a shift in thinking in general, is like previously you would have an old system. And like, even if there were like small bugs or it was really annoying to maintain, sometimes it was worth just not touching because you didn't know what the…
Well, let me tell you, I've been doing this since around 1982. And we still in those days said we're not writing enough documentation. So, I mean, this is a problem, especially as the speed of development increases.
More from The InfoQ Podcast
