Chris Gmyr
Staff software engineer working on Laravel and PHP, currently at Rula and a co-organizer of TrianglePHP. Hosts Slightly Caffeinated, a podcast about Laravel dev...
“If you're going to call something out, validate that you can actually point to the thing that you're calling out. And if you can't, then just drop it. So just that call out increased the correctness of the review probably 15 or 20 points.”
Slightly Caffeinated · Laravel Ecosystem · October 2026
Gmyr and TJ Miller are both building in-house AI pull request reviewers at their companies. His bot kept flagging problems that weren't there; making it cite the exact line of code it objected to cut most of the hallucinated findings.
Transcript
Cool. So last couple of times we talked about a whole bunch of my projects going on, but I wanted to circle back to you and some of the Luma projects that you've had going on recently.
…examples of like, here's, here's what's worth calling out. Here's not, you know, here's stuff that doesn't matter. Yeah, it's, it's so nuanced. And that's been really, really tricky to nail. Like I said, technically, the code and the architecture and all that, not the challenging piece here at all.
Yeah. Yeah, totally. I know we talked about this a little bit before, but basically we're working on the same project, two different companies. But yeah, so I don't remember how much in-depth we went to into last time. But yeah, for the fixtures, I have a whole nother separate repo outside of the PR reviewer, like GitHub action repo to pull in any of these fixtures. So I went through because I've gotten a bunch of feedback from other people around the company in different tech stacks, different teams. And it's like, hey, this PR is just, you know, the review is just not good. And I made comments and it didn't do anything, which it's expected because we weren't pulling in comments and stuff like that at the time. It's just like a fresh eye review, like knows nothing about anything really besides like the diff that we pushed in it. So I've been able to capture, I don't know, probably 20, 25 specific PRs from across different teams and tech stacks and reads of like review and comments and all that stuff and have a bunch of…
Yeah, I really, I really like. So what I had done before is I had it do rule matching on the glob patterns, just how like Claude has it. So it was only adding those to context as needed. And I had something in there before of like your review has to be rooted in rules. So like kind of like you have…
Yeah. Yeah, exactly. So yeah, the agent side is moving. And then I've also tried to make some movements on the human side of reviews as well. So I've been making a bunch of changes to our engineering skills. And there's one in there for like PR create. So if you do this, it'll like open it up as a…
Yeah, we went and made a config file that lives in the root of the repo that has basically glob paths for must-have human review for like auth and like migrations and stuff like that, where it's like that gets called out automatically in the PR, like the agents comment on the PR of like, these…
More from Slightly Caffeinated
