The Case for AI-Focused Human Review
Zach Lloyd, CEO of Warp and former principal engineer at Google, recently published What Comes After Human Code Review? on LinkedIn. It’s one of the more thoughtful pieces in the “code review is dead” genre, because Lloyd isn’t just hand-waving. He proposes a concrete replacement workflow he calls Spec & Verify, and he’s honest about what gets lost.
His framework: instead of reviewing code line-by-line, teams co-author product and tech specs with agents, review those specs as a team, let agents implement, then use verification agents to check the output. Humans review intent and behavior. Agents handle the code. The self-improving review loop he describes, where humans rate agent PR reviews and a scheduled agent incorporates that feedback back into its own prompting, is the kind of concrete mechanism most articles on this topic lack.
I think the Spec & Verify framework is genuinely good. But I think Lloyd draws the wrong conclusion from it. The better version of this idea isn’t removing the human from review. It’s using AI to make the human’s time in review dramatically more effective.
The bottleneck isn’t human review. It’s unfocused human review.
Lloyd describes Warp’s current situation: code review is their number one bottleneck. PRs sit waiting for reviewers. Reviewers spend time reading agent-generated code. Authors pull comments into context and iterate. Multiple cycles. He concludes that the human in the middle of this process eventually won’t work.
But the problem he’s describing isn’t that humans review code. It’s that humans review all the code with equal attention. That’s the bottleneck. An experienced engineer spending twenty minutes reading boilerplate an agent generated correctly is wasted time. That same engineer spending five minutes on an architectural decision the agent got subtly wrong is invaluable.
AI is exceptionally good at this kind of triage. It can surface the parts of a diff that involve architectural choices, novel patterns, security-sensitive logic, or deviations from established conventions, and flag them for human attention. It can tell an experienced reviewer “these 400 lines are straightforward implementations of the spec; these 30 lines involve a design decision you should look at.” That’s a force multiplier. It transforms code review from “read everything” to “look where it matters.” The human stays in the loop but spends a fraction of the time.
The spec verification problem
The strongest part of Lloyd’s framework is the idea that teams should invest more in specifying intent before agents write code. He’s right. Detailed product and tech specs, reviewed by the team before implementation, catch misalignment early and give agents better inputs.
But this creates a new problem he doesn’t fully address: who verifies the spec is correct and complete?
Natural language specs are arguably harder to verify for subtle gaps than code, precisely because they’re ambiguous. Code is at least deterministic. A spec that reads “if the user does X, the app does Y” can sound perfectly reasonable while missing an edge case that only surfaces in production. An agent will faithfully implement that plausible-sounding spec, gap and all. A human reviewer who understands the domain catches the gap, but only if they’re still in the loop.
Lloyd proposes that teams review specs through a “code review type process.” That’s good, but it also means the bottleneck hasn’t been eliminated. It’s been moved upstream. Instead of slow code review, you now have slow spec review. That’s real progress, because catching problems at the spec level is cheaper than catching them in code. But it’s not the automation of human judgment. It’s the relocation of it.
And there’s a subtler issue: the feedback loop between implementation and specification. Sometimes you don’t know the spec is wrong until you see the code behave unexpectedly. Removing humans from the implementation review stage means losing one of the key moments where spec errors get caught.
“Thorough by default” deserves scrutiny
Lloyd notes that enterprise customers find agent code review catches security issues human reviewers miss, and attributes this to agents being “thorough by default.” This is true in a specific and limited sense.
Agents are thorough at pattern-matching known vulnerability classes: dependency issues, common injection vectors, things with established signatures. If there’s a CVE for it or a SAST rule that covers it, an agent will likely catch it, and catch it more consistently than a human skimming a large diff under time pressure. That’s genuinely valuable.
But the security issues that cause the most damage tend to be emergent. They arise from the interaction of components, from architectural assumptions that don’t hold at scale, from trust boundaries that shift when a new integration gets added. These don’t have signatures. They require understanding the system, not just the code. That’s exactly what experienced human reviewers bring, and it’s the hardest thing to specify in a prompt or encode in a verification agent.
AI-focused human review addresses this directly. Let the agent handle the pattern-matching layer. Surface its findings alongside the diff. Then let the human reviewer focus on the architectural and systemic questions the agent can’t answer. You get the thoroughness of the agent and the judgment of the human, without asking either to do what they’re bad at.
The alignment problem hiding in the feedback loop
This is where I think Lloyd’s framework has its most interesting blind spot, and where the conversation connects to broader questions about AI alignment.
His self-improving review loop works like this: an agent reviews PRs, humans rate the quality of those reviews via GitHub comments, and a scheduled agent periodically incorporates that feedback back into the review agent’s prompting (stored as a Skill file in the repo). The agent literally PRs changes to its own instructions based on human feedback. It’s elegant, and it’s a genuine contribution to how teams can work with AI.
But it’s also an alignment problem in miniature.
The loop works because humans provide judgment the agent can’t generate on its own. The quality of the system depends entirely on the quality of that human signal. If humans are thoughtfully evaluating agent reviews, the system improves. But what happens when humans start rubber-stamping agent reviews the same way they currently rubber-stamp PRs? And that’s exactly what Lloyd says is already happening with code review: humans skimming, approving quickly, not reading carefully.
If the feedback signal degrades, you get a review agent optimized for “humans click approve” rather than “the code is actually correct.” The agent drifts toward producing reviews that feel right rather than reviews that are right. And because the agent is modifying its own prompting based on this signal, the drift compounds. The safeguard against this is experienced humans paying close attention to the feedback loop. Which is precisely the thing Lloyd’s framework proposes to minimize.
This pattern shows up throughout AI alignment research. Reward hacking, specification gaming, Goodhart’s Law applied to automated feedback systems. Lloyd’s framework, perhaps unintentionally, is a case study in why human oversight of AI systems matters most at exactly the points where it’s most tempting to automate it away.
The stronger version of the self-improving loop keeps humans deliberately in the review chain, not reviewing every line of code, but auditing the review agent’s judgment on a regular cadence. Spot-checking. Looking for drift. Ensuring the agent’s criteria still reflect what the team actually cares about, not just what gets approved fastest.
Where this breaks down: the legacy reality
There’s a context Lloyd doesn’t address, and it’s the context most working engineers actually live in.
Lloyd’s framework assumes greenfield clarity: you can write a product spec and a tech spec before implementation, and the spec captures the intended behavior of the system. That works when you’re building a developer tool from scratch in Rust with a strong engineering team.
But most software in the world isn’t like that. Most software is legacy systems where the spec is the existing code’s behavior, and nobody wrote it down. I work in environments like this, high-stakes legacy systems where the running code is the only authoritative record of what the system is supposed to do. You can’t write a spec for an agent to implement because nobody fully knows what the current system does in every edge case.
In these environments, human review of agent-generated changes isn’t a bottleneck to be optimized away. It’s a safety mechanism. The experienced engineer who says “wait, that change will break the payroll batch job that runs on the third Tuesday of the month” is providing knowledge that doesn’t exist in any document, any spec, any codebase comment. It’s institutional memory, and it’s exactly the kind of judgment AI can’t replicate because it was never written down for AI to learn from.
AI can still help enormously here: surfacing related code, flagging changes to critical paths, identifying potential side effects. But the human judgment remains essential, not as a rubber stamp, but as the only source of certain kinds of knowledge.
The model: AI handles volume, humans handle judgment
I think Lloyd is right that the current code review process is unsustainable when agents are generating code at scale. He’s right that teams need to invest more in specs. He’s right that verification agents add real value. The Spec & Verify framework is a genuine improvement over the status quo.
Where I disagree is the conclusion that humans should be removed from the review of implementation. The stronger model keeps the human in the loop but radically changes what they’re looking at. AI triages the diff and surfaces what matters. The human reviews architectural decisions, systemic risks, and spec gaps. The agent handles pattern-matching, style enforcement, and known vulnerability detection. Both contribute what they’re good at.
There’s an irony worth noting. The gaps in Lloyd’s framework that I’ve identified here, spec ambiguity, emergent security risks, feedback loop drift, alignment concerns, were surfaced through exactly the kind of AI-augmented analysis his framework points toward. I used AI to analyze the article, identify structural weaknesses, and surface patterns. Then I, a human, evaluated which of those patterns actually mattered and how to frame them. The AI handled the volume. I handled the judgment.
That’s the model. Not Spec & Verify. Spec, Implement, and AI-Focused Human Review.
If you want to discuss: [email protected], BlueSky @tsondo.com, or via the about page.