“Me reading every line would not prevent bugs. In fact, agents reading every line is a lot more effective, not just faster, but more effective in quality than me doing so. I absolutely looked at the code for every PR that I put up, but I was focusing more on big picture stuff. Is the direction of this correct? Is the architecture correct?”
The Peterman Post · Software Engineering & Infra · November 2026
Context
Asked how he handled review on a port of hundreds of thousands of lines that no human read in full, Toub says he checked architecture and direction and left line-level review to agent reviewers.
Transcript
…more agents at it. And we did. Before merge, doing lots of agentic reviews. And then after merge, running agents to continually look for bugs, look for concurrency issues, look for whatever. And they find things, but they never find, it's never, at least in my experience, it's not 100% perfect.
For such a huge volume of code, I imagine. Imagine most of it wasn't human reviewed. How did you handle code review at scale, or did you do any at all?
Yeah. So definitely did some, but I did not read every line. That would be impossible. And if I did, it would also be a waste. There's no, like, me reading every line would not prevent bugs. In fact, agents reading every line is a lot more effective, not just faster, but more effective in quality than me doing so. I absolutely looked at the code for every PR that I put up, but I was focusing more on big picture stuff, right? Is the direction of this correct? Is the architecture of this correct? Is the componentization of this correct? Is the approach, are the algorithms that are being used appropriate? Is there anything fishy about like, you know, if I, a lot of it is like squinting at a 30,000-foot level and looking at the two code bases and does anything look off, right? And then relying on the agents to focus on the line-by-line details. And in many situations, I would sort of do a quick scroll through all the changes to just see if anything popped out. And I might notice a few things here and there. But for the most part, a really good agentic reviewer is going to be way better at sort of the line level details than…
I saw in the blog post that you have this cool bar chart where you basically categorized every single message from you to the agents and what your intent was. Looking over that and reflecting on how you guided the agents, where was most of your time spent? And what were the most impactful times…
There were two or three even higher level categories of things that I was doing. There was one or two categories where I think this just speaks to the state of sort of long horizon tasks with models and harnesses today, where it was me being frustrated with their progress in one way, shape, or…
That's interesting. So your agent will auto-apply and auto-listen on the PR. And if it seems reasonable, it may take action without your input. Is that right?
More from The Peterman Post
