Folkman was explaining why the uptime of large companies looks worse than it has in years. His argument is that their quality did not get worse, they are simply shipping far more code at the same defect rate, so the same percentage now reaches many more people. It is his case for spending AI-won speed on quality checks rather than on shipping again.
Right. And it's not like, to be clear, like, I don't see that happening on our team. I think we've had some not like go all the chain, but there's been things that maybe gotten higher than you'd wanted to see. And I think it's not because people don't want to do quality work. They're feeling like they have this tool that helps them faster. And so they want to keep moving faster. Because now everyone's using it. People also aren't unaware. If you went back like a year and you were doing this, people maybe were like, holy crap, this guy's insane or this girl's insane. Now people are like, that's just AI. Like, I know this is not like thoughtful. So think about it. Use AI to help you do research, to push you to think, have it even give you its ideas. But if you end up in a meeting and your answer is Claude said that or told me that or did this thing, outside of just like normal data gathering, like it's like a code machine or a searching machine for you, that's bad. And I hear that from time to time. People are like, yeah, Claude said it would take this long to build this feature. I'm like, whoa, Claude does not know how long it won't take. And that was more like six months ago that people that I would hear that when Claude was newer on the scene, especially for PMs. I'm like, no, Claude does not. You ask Claude how long it'll take six months ago, and it's like, oh, a week. And they'd say, go build it, and it'd be like, built.
It was crazy. It would overestimate everything. Okay. So I think I got a good sense of what loops are, what hooks are, when I need to use them. Talk to me a little bit about from product to engineering. What engineering loops should be out there? What engineering hooks should be out there? And then PMs, like interfacing with engineers. When does the PM work? PM loops end and the engineering loops begin.
Some of the most important loops for engineers right now are quality loops. The thing that people don't talk enough about, in my opinion, is if you ship, let's say, twice as fast with AI and your quality rate maintains the same rate. Let's say you have a 1% bug defect ratio or something. Your customer experiences twice as many bugs if your quality doesn't improve. The customer doesn't experience your defect rate. They experience the number of defects that you push out there. And so we're seeing this. I mean, you look at big companies, their uptime is the worst it's been in a long time. And my opinion is it's not that their quality got worse, actually. It's that they're just. Just shipping more code. And so more things go wrong at the same rate. And we've seen that. And so we've really started and tried to invest in quality improvements through loops for engineering, like our standards, checks, automated end-to-end type testing. There's a billion things you can do there. But if you don't have AI working on the quality side, you're going to move faster and your customers will feel like you've gotten worse at building things, even if you didn't, because they will experience the velocity of more bugs. So that's where I would start. If you're looping, loop on quality for engineers.
And should PMs be pushing PRs? Where does that go till? Should PMs be working on engineering tasks at all? I
think it's definitely appropriate for PMs to work on engineering tasks where it makes sense. Like, let me give an example. I don't know if I'd put a PM working on our back end billing system and making upgrades to it that could impact people's money flow and all that. There's a lot of like important decisions to be made there that are more architectural. I think it's fairly reasonable to push some front-end changes where we have good decoupling from the back end, good APIs, where we have good CI/CD, good quality testing. The more you trust your system to do a lot of that checking, the more I think it's okay for anybody, UX, PM, Eng, to push. And so like I kind of mentioned before, you are at the mercy of your systems before AI. Great systems do really well with AI. If you're a company that had really bad systems and relied a lot on humans and slowness to kind of protect you, you don't want PMs coding in there because you barely want probably your engineers coding in there. But great systems, I think there's no reason PMs can't jump in and help. And the one thing I would be careful of is make sure it's agreed upon in the team because it can create some shadow work for engineers because you want code reviewed. That's the same for any engineer. And I've seen PMs be like, yeah, I just pushed this. I was working on the side. Can someone take a look? And now you've generated maybe hours of work for someone that wasn't planned on. And you might get frustrated because you're like, hey, I'm trying to ship stuff. And they're frustrated because it wasn't like agreed upon that that would be part of their workload. And as an engineering community, this is one of the things we're struggling with: how do we manage the code onslaught? Because you could have AI generate basically infinite code. So how do you do that? Systems is one thing people are thinking about, but the dirty truth is most companies' engineering systems weren't perfect before. So they're not perfect now. And I tend to push quality, like I said, on the loops because I almost think that the quality advantage of AI can beat the velocity advantage because it lets you ship safer.
Okay, so that's one part of it, which is PMs doing engineering work. It sounds like PMs shouldn't be trying to become engineers, but if they're doing some front-end changes where, again, they have put in probably more time on it than the reviewer would need to, then it might make sense. What about the other side, which is engineers doing vibe PMing?
