Connect your Claude, Codex or OpenClaw for full transcripts on millions of podcasts. Two week free trial, cancel anytime.

Start free trial →
JS Jonathan Stark Jonathan Stark Expert on value-based pricing and founder of The Value Pricing Library.

“It is totally true with Claude, or any of them, any of even the most advanced ones. It is so bad at stuff that I'm good at. So why do I think it's so good at stuff I'm bad at? Because I can't tell the difference.”

Ditching Hourly · Agency Business · November 2026

“It is totally true with Claude, or any of them, any of even the most advanced ones. It is so bad at stuff that I'm good at. So why do I think it's so good at stuff I'm bad at? Because I can't tell the difference.” — Jonathan Stark, Ditching Hourly

Context

Jonathan Stark applies the Gell-Mann amnesia effect to AI: you only notice a model's mistakes in the areas where you are the expert.

Transcript

Ditching Hourly Around 37:36 into the episode
Jonathan Stark

…it recently where when I was a little kid, I went to a rock show, like a paleontology kind of rock show. And because I was a nerd even then, and a reporter was there with a photographer, and they interviewed me, took my picture, put it in the paper the next day, and it had like six major errors.

Dave Mabe

Yeah.

Jonathan Stark

And I was like, I didn't get the, I think, I think it's Gelman amnesia. I don't remember the exact name. I didn't get Gelman amnesia. I never trusted the newspaper again. But most people are like, oh, this one article in the newspaper I'm reading that I know a lot about is totally wrong. It's completely, or at least it's 101 level, simplistic stupidness. But all these other articles are expert level genius articles that don't have typos or don't have the names wrong or don't have a correction in tomorrow's edition, which I'll never see anyway. And I mean, when you're on a daily deadline, and I'm from a newspaper family, a lot of stuff's wrong. But anyway, that's the, that's the, I think that was the original use case, but it's the exact same thing or the original impetus for that study was newspapers. But now it's, it is totally true with Claude and or any of them, any of even the most advanced ones. It is so bad at stuff that I'm good at. Yeah. So why is it, why do I think it's so good at stuff…

Dave Mabe

Yeah.

Jonathan Stark

Anyway, so I'm just sort of, I'm sort of singing your song here. So what's the, what are the common failure modes for folks you're not just in general, but folks you actually work with, like folks you talk to, what are the failure modes for people who are, you know, pretty serious and maybe have…

Dave Mabe

Yeah. So by far the most, the biggest mistake I see or the most frequent mistake I see now is people using Claude to create their own backtesting software. So there's lots of backtesting software out there that works. And a much better approach is have teach Claude how to use the existing…

Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.

More from Ditching Hourly