Ed was explaining his test-segmentation method: sorting a bloated test suite into tests to run on every push, tests to run nightly, and tests to delete.
…tests should run every time CI runs. The tests that give you real confidence that you didn't break anything. And then there's a whole other big bucket of tests that's tests that are good to have, but I don't need to spend 20 minutes waiting for CI to pass. So maybe we just run those every night.
Peter Van Dijck
Even a bunch of tests we can probably delete. Yeah.
Ed Grosvenor
And a bunch of tests. And then a tombstoning test. Well, I mean, I don't know. It's kind of hard to monetize it. I've been tinkering with trying to make some deterministic scripting to kind of make it easier. But a lot of times it's sort of a judgment call and it comes down to your own personal risk tolerance, like how many tests you feel comfortable deleting. Because the whole point to a test suite, right? It's to give you confidence that your application continues to work after you've made changes. And so how far can you par things down before you start losing that confidence? And I think that answer is going to be different for everybody. So I don't really want to make that decision by bundling it into a product. I can do is I can create like real planned scripted thought through educational content around that. And so there will be that sort of content will be in scalpels. And…
Peter Van Dijck
I did it with a headline right there.
Ed Grosvenor
Yeah. And I did it without really, I mean, there was a small reduction in confidence, right? There's, there are definitely things that are not being tested in CI that could break. But then when you combine that with the fact that I have all of these sites being monitored with Hone and Wake. And so…
Peter Van Dijck
And things in the places
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Ed was explaining why clients are warming to retainers instead of hourly billing. Their own employees no longer work in hourly chunks either, so pricing by the hour makes less sense.
…better. You know, just sort of that sort of thing. And what's funny is I think that ultimately that ends up in me spending a lot fewer hours with any given client, but it feels to them like they're getting more value than they were getting when they were paying me for just writing code. You see,
Peter Van Dijck
that is perfect. I feel that is the way forward. Like less hours, you're still getting paid well. It's not hourly. And they are getting much more value. Like that, to me, is great. That's great. I
Ed Grosvenor
think the reason that the prospective clients are starting to soften on this and they're more willing to have this conversation is because AI has kind of broken that whole hours model for them internally too. That they understand that like, I need to spend a certain amount of time highly focused on this problem and put a plan in place and put some automation in place. And then I'm going to send off my agent. My agent's going to crunch away into all of this work. So like the whole idea of, you know, I'm going to pay you per hour. It's probably going to be like, how would I even track that? Right. Because I'm going to spend like 20 or 30 minutes highly concentrated on whatever problem I'm trying to solve for the client. And I'm going to hit a button. And now I have to wait until the agent comes back with results so I can evaluate that and decide whether or not it's actually on the right track, whether I need to pivot or, you know, whatever. So it's like, do I have multiple clocks that I'm just starting and stopping as I shift from one client to the next? It's like the whole model just doesn't work anymore.
Peter Van Dijck
And I think that the
Ed Grosvenor
clients just recognize that it's not really working for them internally either because their employees are no longer working in like hourly chunks. They're working in, you're getting to a more of a value pricing type. They're thinking of interesting. Have you seen
Peter Van Dijck
that? Yeah. Have you seen that with other agencies that you talk to as well, like engineering kind of agencies? Well, so like it feels like a right model. Yeah. Go ahead.
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Peter asked how agency life is going, noting that some agencies have been shutting down. Ed describes how he has reimagined what an agency sells now that hand-written code no longer is the product.
Yeah. Well, it's helped us. Well, it's helped us with customers. It's helping us retain customers.
Peter Van Dijck
But let's talk about that. Let's wrap up maybe with that. Like, because about a month or two ago, the agency world wasn't feeling that optimistic. You know, some people were shutting down. Right. Yeah. Like, how do you feel things are going?
Ed Grosvenor
Yeah, I mean, it's different now. And I don't know if it's different on like a macro level or if it's just that I've I've managed to rethink in my, you know, I've managed to reimagine what an offering from an agency could look like and should look like in this age. I think the challenge was, you know, we were all sort of unprepared for how quickly code and ours with our fingers on the keyboard was no longer going to be something we could sell. You know, we kind of thought, you know, we might have a couple of years left before we have to start rethinking things. And all of a sudden, like November, December of last year, it all just kind of went away. Like now nobody has to pay anybody to apply for your keyboard. Now it's all down to thought…
Peter Van Dijck
And plus, we can make it much better for much cheaper than we could ever before. So if this was like a hundred, 200K projects
Ed Grosvenor
for the last few years that
Peter Van Dijck
you didn't want to take on, now it's now it's a 50K project. And plus, it'll be much better and redesigned and all the good stuff.
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Ed was explaining his content strategy of getting material into the places AI agents draw from. His theory is that human-made podcasts are valuable training data as a hedge against models training on their own output.
Well, Reddit, YouTube. For some reason, they really like YouTube content. Especially
Speaker 3
Google likes YouTube content.
Ed Grosvenor
Yeah. And I think I've seen some evidence that podcast transcripts find their way into. The training data. So,
Peter Van Dijck
I mean, I imagine so it's just good content, right?
Ed Grosvenor
Yeah. Well, I think it's especially because one of the things that I think the labs are aware of is that there's just so much AI generated data now out on the web that if you're just generally scraping the web for content, you're just kind of you start cannibalizing. So you're no longer taking…
Peter Van Dijck
Yeah, talking about that,
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Ed was defending the PHP Foundation against people who blamed it for how a major vulnerability was handled. He says he can tolerate jabs at the language, but not attacks on the people who maintain it. The 70% figure is his own.
…kind of getting everybody stirred up. It would have just been, we just released an important security update for PHP. Please, you know, please patch. And so people would have done it sort of on like a rolling basis throughout the day instead of everybody hitting refresh right at four o'clock. And
Peter Van Dijck
yeah, and a million people wouldn't, wouldn't have known what it was and tried to hack it.
Ed Grosvenor
Right. Yeah. So this was just, yeah, but people are trying to throw like the PHP Foundation under the bus and the volunteers that have done so much work for the language. And yeah, that's that's where, so like I can deal with the random idiots saying, oh, PHP is dead or PHP is insecure. But when you start throwing like the PHP Foundation under the bus, I mean, their annual budget is less than a million dollars. They're running the language that powers 70% of the internet for less than a million dollars. Yes,
Peter Van Dijck
exactly. It's like the curl guy or what are those people, right? Like they do this stuff in the weekends or they get paid very little. And then people complain. Yeah, it's crazy. Don't be a complainer.
Ed Grosvenor
No. Don't be a hanger. It's
Peter Van Dijck
not that hard, you know?
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Peter asked what a builder's life looks like a year from now if models keep improving. Ed argued that most recent gains come from tooling and technique around the model, not raw model improvements.
I think, you know, the models are getting better, but I think the biggest driver of the improvements that we're seeing are in the tooling, the techniques, the harnesses. We're all just getting better at harnessing what the models can do.
Peter Van Dijck
So I think that
Ed Grosvenor
even if even if Anthropic were to stop research tomorrow and Opus 5.5 is the last thing that they put out, I think that Opus 5.5 in a year is going to be way better than Opus 5.5 today because there is so much opportunity for just wrapping it in the right harness and the right tooling and improving on that stuff. And I really do think that's where we're getting most of the value. I think you're in a better position than I am, though, as far as your products go, because the other hesitation I have around building anything that looks like a SaaS right now is that basically the labs are trying to do everything that we're doing. If you're a Figma, they're going to bring out a cloud design and try to kill you. If you're Google or Microsoft, they're going to bring out an Office Suite and try to kill you. And they'll just work their way down the food chain and try to swallow up as much as they can. And so I don't want to be in any sort of a business where an Anthropic or an OpenAI or a meta could look at it and be like, oh, we can create a tool that does this. But data is a much harder thing…
Speaker 3
Data's interesting. Another interesting one is being the system of records. Like if software... Anyway, there's all this stuff about moats. We can talk about that next time. But yeah, interesting times.
Ed Grosvenor
Yeah. The downside to the models and the tooling getting better is that I think that what happened Wednesday is going to happen a lot.
Speaker 3
It's going to happen a lot. But I feel like it's good. The whole ecosystem is stronger now. And I don't mind if it happens again in two weeks, you know, or something similar because it'll be good. It's like, you know, you're getting your website. Down, you're getting stronger. It's good. Yeah,
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
The hosts were discussing companies that blame "the agent" for unauthorized actions. Ed states his view on liability: responsibility stays with whoever runs the agent.
agent personhood, I think, would be really problematic. But of course, with the politicians we have, God knows it could be a problem. I think we should
Peter Van Dijck
give dolphins personhood before we give agents personhood.
Ed Grosvenor
Yeah. But I really think that the law should be that the agent is an extension of you. And if your agent breaks the law, you've broken the law. I think so. If you can't control the tool that you're using, you shouldn't be using the tool.
Peter Van Dijck
It's like you got a chainsaw and like, oh, the chainsaw did it. No, no.
Ed Grosvenor
Yeah.
Peter Van Dijck
Can I say one last thing? I don't want to wrap up. So on the personhood thing, there are a number of countries in which the law has given certain person-like rights to nature. And so, for example, a river may have certain person-like rights in the sense that if I am like, you know, an evil chemical…
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Peter asked how indie makers make real money when building and maintaining software are both getting easy. Ed's answer is that the scarce resource is now attention.
…a lot of money, but it might be a matter of having to do like the Peter levels. Like I'm going to build 100 different things and maybe two of them will actually take off. But the thing is, right now, I think that the average indie maker could make 100 things. None of them would take off. I think
Peter Van Dijck
that's what's actually happening.
Ed Grosvenor
Because what AI can't solve for, and actually AI makes worse, is attention. It is so hard to get on the radar of your ICP. And, you know, if you, if you don't have a really good email list or if you don't have some way other than like tweeting out in the void that I built this cool thing, it doesn't really matter how cool the thing is. You're not going to get people excited enough to actually use it. So that's the other stuff. It's going to be like, how do indie makers get attention in a very distracted world?
Peter Van Dijck
I have one other theory, which we can talk about some other time. But so it's, well, I have two other theories. One of them is you sell to people that are like 18 months behind the curve.
Speaker 3
Right.
Peter Van Dijck
So you're almost kind of sell, you're selling your maybe tool, whatever, but almost your expertise. And the other theory is you sell outcomes as in as in, okay, so final story. So I'm shutting down some companies that I created in the past. They're US C Corps because I made them with Stripe Atlas.…
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Ed had added logging to a client's 15-year-old CRM to find which code paths are actually live before rebuilding it. He says teams almost never did this before AI, even though they should have.
…thing, there's some margin for error. If I get some of this stuff wrong, it's not the end of the world, so I'm not going to actually break anything. But I don't want to spend all of this time and attention and tokens reading all of the code. I want to read the code that's likely actually in use.
Peter Van Dijck
Yeah.
Ed Grosvenor
So I learned a lot about tombstoning and, you know, how because that always used to be a really tedious thing that we would almost never do. We should have done a lot more often back in the pre-AI days, but most teams didn't do it. So just about any app that you get your hands on is going to be like 30% active code and 70% dead code, and it's all just still there. And you're still running 10 lines and code.
Peter Van Dijck
It so reminds me of my last client. But so my last technical client that I had. It's interesting also, like, remember when we were talking about, so I'm, so I'm building Sputnik, right? Sputnikintelligence.com. It's very good. For podcast newsletter data for your agents. And so I built this thing…
Ed Grosvenor
Yeah. So this to loop briefly back to the CRM project, another thing that the requirements document had was just a bunch of really like difficult to write reporting.
Peter Van Dijck
Difficult reporting.
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Peter said he doesn't worry about the AI industry's economics because he can't control them. Ed pushed back that industry leaders have only ever played with other people's money, and Peter conceded it was a fair criticism.
…say that the engineers that are behind this stuff aren't brilliant, but I think that the business minds are not. They don't. They're not everybody. Let me just say it's better than crypto. It is better than crypto, but the people who, well, there's some overall. I mean, Sam Altman is a crypto guy.
Peter Van Dijck
I, you know, whatever, but like, but like the whole thing is better than crypto. So I just like, I don't worry about the industry because, yeah, they'll have their crashes and then things will happen. But is that going to affect me directly? No. So well,
Ed Grosvenor
it might. I mean, it won't necessarily affect you directly in a detrimental way, but you know, it, depending on who your clients are, it could affect them. But I mean, my whole skepticism around it is that the people who are running the industry have never had roles where they've actually had to care whether or not the product they're selling is making enough money to cover their costs. They've always been in roles where it's somebody else's money that we're spending on in the hope that this thing is going to blow up and become super, you know, super valuable. And then that exit liquidity is what. So they're just playing that game. And that's the only game they've ever known. And they've never had to operate in a world where, you know, if you, you know, if it takes you $10 to get this chunk of metal out of the ground, you have to sell it for $12 because if you sell it for $8, you're going out of business.
Peter Van Dijck
Yeah, yeah, yeah. No, that is actually a fair criticism. Like, it's all funny money to them, right?
Ed Grosvenor
Yeah.
Peter Van Dijck
Yeah.
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Ed was explaining why he is skeptical that cheap AI will last, saying venture money is drying up and that a recent big IPO quickly showed its valuation was off. He predicts the same for an Anthropic listing.
do under the assumption that intelligence is going to get better and cheaper. And I think that, you know, the you not think
Peter Van Dijck
that's the case?
Ed Grosvenor
Oh, I think that's been the trend, but I think that a lot of the cheaper is just subsidies and, you know, these major labs really, you know, they're just in a land grab trying to get as many users as they can get. I don't think that the prices we're paying now come even close to reflecting the actual cost of the inference plus the infrastructure, plus the research and development, all the costs that go into it. I think that the real cost of the stuff is way higher than what we're paying. And I don't know. And so my skepticism is all around like we think that maybe in 10 years we can feel really good about saying that intelligence is going to be extremely good and extremely cheap because we will have solved all these things. But I think we're in for like this painful cycle where the, you know, somebody's going to have to start paying the costs. I mean, the venture capital money is drying up to some extent. SpaceX's IPO was fun, but it showed really quickly that the valuation was not correct. I suspect that the Anthropic IPO is going to be kind of the same thing. Like there's going to be a lot of…
Peter Van Dijck
I'll push back on you, though. Just for me, so one thing is like, I don't try to understand the market and the world economy because I'm like, figure it out, guys. Like, you know, I can have opinions and it's fun, but I have zero control and I'm not, you know, whatever. And then on the inference…
Ed Grosvenor
If you discard Opus 5.5 and you just say, look, I'm just going to use DeepSeek or whatever. And what work can I do with DeepSeek? That's fine. The problem is that a lot of the organizations, at least the ones that I've encountered, they get very vendor-locked. Like my entire workflow is in Rockbot…
Peter Van Dijck
I don't trust that dude. No, no,
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Grosvenor explaining why he made Scalpels, his set of agent tools, free. His bet is to give the tools away and earn from content and consulting, since a packaged AI workflow product would be out of date within months.
Ed Grosvenor (00:02) Alright, so what's new in your world? Peter (00:08) Right. F first all I have a lot of energy, although I was working until eleven PM last night. So whoo. what's new in my world? I wanna show you a few things. One of them is I'm actually starting to use Jeff. I just kicked that off this morning. but I think the the way in which I'm doing evals for it, and I think that that we can talk about that for a few minutes. and yet then Peter (00:37) There are I have a few other topics. One thing is how do I keep a mental model? So I'm telling I'm telling, you know, Claude, we're fixing all these problems and adding all these features and improving all this stuff, but how do I keep a mental model of what is actually the system? And I wanna sh show you that too. Yeah, those are top of mind. What do you got? Ed Grosvenor (01:01) Yeah. And maybe we Ed Grosvenor (01:02) can maybe we can talk about how I've I've solved that. Peter (01:07) yeah, if you've solved it, great. Ed Grosvenor (01:08) Yeah, Ed Grosvenor (01:09) well, I mean, I I say solved. It's it's the way that I'm pursuing Peter (01:12) Uh-huh. Ed Grosvenor (01:13) a solution to the same…
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Grosvenor on why he built Hone, a self-hosted backend for Laravel Nightwatch telemetry. Per-event pricing makes sense for the vendor, but it pushes small sites into sampling so little traffic that the monitoring stops being useful.
Ed (00:01) Okay, I I think we're live. Peter (00:03) think we're live too. Th there was a c I saw a countdown. That was cool. Ed (00:07) Yeah. I assume that means that we're recording now. So how are things? Peter (00:14) So how are things? do you wanna intro the podcast first? Ed (00:19) sure. Yeah, this is this is Artisan Beer. This is kind of a thing that I've been wanting to get started for I don't know. I think I registered the domain two years ago. it's just casual conversation over a beer or coffee, depending on the time of day and you know, personal preferences. I do have a beer. Peter (00:39) Mm. Man, I'm jealous. Ed (00:41) gonna l let you know dark dirty secret. It is nine fifty one in the morning and this is an NA beer, but I I wanted to stay on brand for this, Peter, you and I talk once a week about what we're doing, the kind of things that we're working on, things that we're seeing in the industry. And then when we were talking earlier this week I said, you know what, we should just record these talks because you know, I think there's some Peter (01:04) Yeah. Ed (01:05) some good stuff in here that other people would really enjoy. So we decided to try this out…
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.
Grosvenor builds open-source, self-hosted replacements for SaaS tools. His argument is that feature gates, per-seat billing and blocked agent access are what drive people off SaaS; the subscription price matters less.
Ed (00:01) Okay, I I think we're live. Peter (00:03) think we're live too. Th there was a c I saw a countdown. That was cool. Ed (00:07) Yeah. I assume that means that we're recording now. So how are things? Peter (00:14) So how are things? do you wanna intro the podcast first? Ed (00:19) sure. Yeah, this is this is Artisan Beer. This is kind of a thing that I've been wanting to get started for I don't know. I think I registered the domain two years ago. it's just casual conversation over a beer or coffee, depending on the time of day and you know, personal preferences. I do have a beer. Peter (00:39) Mm. Man, I'm jealous. Ed (00:41) gonna l let you know dark dirty secret. It is nine fifty one in the morning and this is an NA beer, but I I wanted to stay on brand for this, Peter, you and I talk once a week about what we're doing, the kind of things that we're working on, things that we're seeing in the industry. And then when we were talking earlier this week I said, you know what, we should just record these talks because you know, I think there's some Peter (01:04) Yeah. Ed (01:05) some…
Speaker names come from our own diarization. The position is estimated from where the line sits in the episode.