“It is actually quite difficult to get the AI to say, I don't have that information. That information doesn't exist. That scenario doesn't exist. So what it's going to do is invent an answer, right? Because it's designed to produce content. That's what it does.”
Pragmatic AI with Matt Stauffer · AI Builders · November 2026
Context
Matt Stauffer said people turn to AI for docs because it can answer their exact scenario. She replied by asking what happens when that scenario isn't in the docs, as her case against trusting confident-sounding AI answers about technical content.
Transcript
…and it gave me an answer right away and it gave me exactly the answer that I asked for. And it looks good. So there's a very, very big disconnect between what is the job of somebody producing technical content and why are you being, it's not disintermediated, right? It's like re-disintermediated.
…that you mentioned was you said it's a failing of the original content. So is there a vision that you have or would have had in the past for how we should differently either write or present content that would have not necessitated the AI replacement? Or like, what's the solution to that problem?
It's a great question. So first, if you ask the AI Fails super specific question, then what you want is for it to return the exact answer.
Yeah.
And if you set things up correctly and if you have, you know, kind of the right guardrails around your AI, you can set it up in such a way that when you ask that, I have X and then I configured Y and then Z happened and it says, oh, here you go.
Okay.
More from Pragmatic AI with Matt Stauffer
