“Now when AI is changing the tests and adding new tests, I completely lost track of it. Like we started a new code base. On a few weeks, we had 200 tests. Now I think a couple of months in, we have 800 tests. I don't trust it at all.”
Maintainable · Hands-On Engineering Podcasts · October 2026
Asked whether she has worked on projects where a big test suite still left the team nervous about changes. Moreira says it is happening right now on an AI-assisted codebase, where she would have to redo the tests to understand what they actually cover.
Transcript
…with confidence. In a code base I've worked for a while, I can like know, okay, this is like super low risk. I can like make changes here, I can introduce new stuff, that's fine. And that other part, like if people ask me to make any change there, I'm like, let me take a while and think about this.
Have you found that you've been part of projects that there might have been a lot of tests, but maybe you and or your teammates still were a little bit nervous to make changes in areas of your code base?
…from learning and from short experience, I've been thinking like, I have thought during working with this code bases that like, oh, I'm not going to make any changes unless I really improve the test suite and then I can make changes, right? That's how I guess you refactor legacy code. And so legacy code, yes, a little bit of that. But more recently, I feel like this problem has exploded because now when AI is changing the tests and adding new tests, I completely lost track of it. Like we started a new code base. On a few weeks, we had 200 tests. Now I think a couple of months in, we have 800 tests. I don't trust it at all. Like every time I'm testing things by hand, they don't mean anything to me. So I know if I want to really have confidence in one part of the code base, I would have to completely redo the tests. And to get started, just understand what we are testing, right? What scenarios are we testing and which ones we're not testing? And then I can have confidence. But if I don't know what I'm testing, then…
all. If it's generating a lot of code and you're saying, I don't have a lot of confidence in the tests that it's been producing and you have 800 tests now and you're like, I still feel like I need to do a lot of manual testing. It kind of, to me, reminds me of being dropped in a project that I…
I think it's very different. Like when you join a new project and people have written the tests before, I still have much more trust in those than in the tests that AI writes now, right? Because people have wrote them and there must have been a reason. For them to write it. So they either were…
That makes a lot of sense. Well, I think before we go too much further, I think I want to give listeners some context about the type of work that you're doing. So I know you've spent several years now helping develop Quint, and you became, I believe, CEO earlier this year. For someone hearing about…
More from Maintainable
