DS Dave Smith Dave Smith Software developer who co-hosts Soft Skills Engineering, a weekly advice show for software engineers.

“There's one really simple heuristic, which is how many prompts did it take to get the document to its current state? And if the answer is less than, I don't know, 20, then you probably have a crap document when it comes to design docs, planning docs, large ticket planning docs, things like that.”

Soft Skills Engineering · Hands-On Engineering Podcasts · October 2026

“There's one really simple heuristic, which is how many prompts did it take to get the document to its current state? And if the answer is less than, I don't know, 20, then you probably have a crap document when it comes to design docs, planning docs, large ticket planning docs, things like that.” — Dave Smith, Soft Skills Engineering

A listener had written in about coworkers sending him AI-written pull requests and planning documents to review. The hosts were trying to work out how a reader can tell whether a person put real thought into an AI-assisted document.

Transcript

Soft Skills Engineering Around 17:00 into the episode
Dave Smith

does it do what the spec says to do? Which this question asker says it does not. Yeah. And that's just bad engineering, regardless of whether it's AI or not, if that's the case. Yeah. I mean, by the way, I think you're right, Jameson, that this question asker is indeed, does indeed carry an expectation that the code will look like the code that we used to review five years ago. You know, carefully thought out, human-authored. But I don't know how else to say this, but we are walking into a tidal wave right now that is washing up on shore. And that tidal wave is AI-generated code. And I am convinced that we cannot oppose this tidal wave. We have to find a way to live with this volume, with this new way of writing code. And we cannot insist on the old way of carefully reviewing every line and having it feel like human, curated, carefully written, so that a human can read it. It's just, we just can't have that anymore, I think. Whether we like it or not. You know, it's just, I don't think we can have that. So I have to ask, how do we learn to live in this new world? I

Jamison Dance

don't know if I could clearly articulate what good AI planning output looks like. I feel like that's something we need to figure out if we're going to use AI to author not just the code, but these kind of meta documents. I don't know. I just quickly jumped to, well, obviously this was written by AI, and therefore it has all of these problems. And unless you rewrite it, I don't know how you get rid of the it was written by AI sense, even if you are able to make it good, you know? Like, so I think we have like a signal problem to show, no, no, no, like, yes, this was produced with AI, but also I have, I have worked on it enough with my own human brain to put my stamp of approval on the output.

Dave Smith

Yeah, I think there's one really simple heuristic, which is how many prompts did it take to get the document to its current state? And if the answer is like less than, I don't know, 20, then you probably have a crap document when it comes to like design docs, planning docs, large ticket planning docs, things like that. Maybe, maybe 10 to 20, somewhere in that range. But that may be what needs to happen here. We need some kind of mechanism that says this was a one-prompt document.

Jamison Dance

Yeah, we need to like, there's some new crypto algorithm. I mean, cryptography, not cryptocurrency. Yep. Although I'm sure, sure, put an NFT on it, I

Dave Smith

guess. Then it'll be worth something. Yeah,

Jamison Dance

some new signature you add to say, like, I assert that, I don't know.

Speaker names from our own diarization · position estimated from where the line sits in the episode

More from Soft Skills Engineering