The Pragmatic Engineer · Hands-On Engineering Podcasts · November 2026
CodeRabbit's CEO claims his team's defect rate is falling, not rising, as AI writes more of the code. He says he hasn't measured it rigorously yet, and the condition matters: AI does better when it can validate against something.
Yeah, because they're like, as the saying goes, like, there's only maybe one way of building Unix. If you were to build it today, you're going to come up with the same abstractions.
…good documentation. And I guess when you have something like this, and there's only, I guess, I assume there's only one or maybe two ways to do things, but when you have this, agents are pretty good. When there's good documentation, no matter how complex it is, they can navigate the complexity.
Those kind of things they're like great at. Like things like, okay, I'll optimize my Docker files. They can just keep running it against Orbstack locally and optimize it. Optimize my CI cache or do this bund migration. I mean, those kind of things are super easy right now. I mean, if you have good unit tests and testing, like you can validate that, hey, things are coming up correctly and the smoke tests are passing. You can even try on production. And usually things are working. It's not like we are shipping a lot more defects. Our defect rate, I would say, is going down. I mean, this might be a hot day, but like humans were writing slop, not AI. I mean, that's what I'm realizing. Like, AI is actually writing less slop at this point if it can validate against something.
So you're saying that in the past, we ship, like most teams would ship kind of slop or low quality stuff when we didn't validate it. And to be honest, if we're just being honest, we didn't always validate it, right? We had more outages
in the past. I mean, that's my observation. I still have to measure it, but I would say when we were shipping with hand. And
you're talking about just code. Rabbit, right?