A controlled run shows what happens when multi-round AI code review never reaches a fixed point, and why "non-convergence" is a signal teams should treat as a stop condition.
The standard architecture for multi-round AI code review assumes convergence over time. Reviewer agents identify findings in the code, which the implementer revises. After a few rounds, the findings list shrinks to zero, or close enough, and the pipeline ships the code. The implicit promise is that more rounds produce code with fewer outstanding issues.
We ran a controlled experiment to see how that promise holds up when reviewers disagree about how to fix the problem they all see. The result? After three rounds and roughly half an hour of agent time, the central security issue remained unresolved, the count of findings had grown rather than shrunk, and the loop exited because it hit its round budget, not because anything had been fixed. In the process, we learned a lot about a failure mode that bounded review loops do not handle well by default.
The AI code review setup
The feature was an updated, secure workflow for changing the email address on a user record. The requirement stated that the application must confirm the user’s password and current email before updating to a new one. The workflow is typical for many web applications: the user verifies their current password, the system sends a confirmation token to the existing email address with a 24-hour expiry, and the email change applies only after the user presents the token.
The implementer agent built the confirmation flow as specified; however, it also left the existing endpoint in place. That endpoint, PUT /api/user, accepts and immediately applies changes to any property of the user object, including the email address. While the new secure confirmation path was properly implemented, the old shortcut remained, serving as a simple bypass of our new security control.
The agentic reviewers caught the exposure. In the initial review round, the security reviewer flagged the business-logic bypass as a single blocking finding. The mitigation was conceptually simple. The updated user function had to stop accepting an email field and instead route all email changes through the new confirmation flow.
The first round of iteration did not produce that fix. Instead, the implementer added a hard rejection that returned HTTP 422 (Unprocessable Content) for any PUT /user request containing an email field. That broke existing callers that were sending back the full user object as part of normal update flows.
The broken logic was identified and repaired in round two, after which review round two re-flagged the bypass. This time, four reviewers (security, access control, architecture and code quality) raised it again, with seven blocking findings.
Review round three never resolved it. The pipeline exited at the round bound with the bypass still in place. Final state, 37 findings, 7 of them blocking, all left unresolved in the run record.
Why AI reviews do not converge
The reviewers all agreed that the bypass was real. They disagreed on the fix language. One said to "reject" unauthorized email updates. Another said "strip" email from submitted user record updates. A third said "ignore" email updates. Each phrasing implies a slightly different code change. Rejecting the request returns an error. Stripping the field accepts the request but drops the email. Ignoring the field accepts and applies the request but skips the email assignment.
The implementer agent received these as parallel findings in a flat list, with no shared vocabulary for the mitigation. It chose one approach in round one, broke unrelated functionality, and could not produce a single fix that satisfied all three phrasings simultaneously within a single revision pass.
The deeper issue is that the fix required a cross-functional invariant. The updated user function had to be aware of the new confirmation flow and behave consistently with it. That kind of invariant is straightforward for a human developer who has built up a mental model of the code. It is harder for an agent revising one file at a time, where each pass treats the code as an isolated text edit rather than as a participant in a system-level contract.
What this says about bounded AI review
Most AI review pipelines treat the round bound as a quality guarantee. After N rounds, the code is "ready." The run we observed suggests that bound alone does not tell you whether the code works as specified or meets quality metrics. It is telling you the loop hit its budget.
The right interpretation of a bounded review loop aligns more closely with how clinicians interpret a treatment trial. Most patients in a successful medical trial respond to the treatment by the trial's planned endpoint. Some patients, however, do not. No medical researcher would assume that patients who didn’t respond to their treatment were cured; instead, they are considered outliers that warrant further analysis to understand why. A non-converging review run is the AI-pipeline equivalent of a non-responder. Treating it as "done" because the loop exited is a dangerous read.
Three practical implications:
-
First, instrument the loop to detect non-convergence. The simplest signal is tracking the number of findings as it changes from round to round. If the number of findings is flat or increasing, the loop is not converging. If a specific finding has been raised in two consecutive rounds, that is also a non-convergence marker on that finding.
-
Second, require human escalation on non-convergence. If the count of blocking findings has not strictly decreased between the last two rounds, do not ship; hand the run to a human. The cost of escalation is far less than the cost of shipping unresolved security issues, especially ones that the pipeline visibly knew about.
-
Third, treat cross-functional invariant work as a specific convergence risk. In this run, the finding required code in one function to behave consistently with a security control implemented in a separate function. That kind of dependency does not surface in a flat list of findings, and it is not something an agent resolves cleanly by revising one file at a time.
What this does not say about automated review
The run does not say that a multi-round AI review is unreliable. Most runs in our broader matrix converge cleanly, and the loop performs exactly as designed. The problem is that a loop exiting on budget looks identical to a loop exiting on convergence. Without instrumentation to distinguish the two, the pipeline has no mechanism to surface the cases where it failed.
It also does not imply that you need unusually complex logic for these cases to occur. The email-change confirmation flow is well-trodden ground. An agent that can build the flow correctly in round zero can still fail to repair the business-logic bypass when the reviewers cannot agree on the fix language. The complexity is inherent in the review loop, not in the feature.
Conclusion
The instrumentation required to catch this is not expensive, and the failure mode it prevents is not subtle. A bounded review loop with no convergence test runs to budget and exits. Whether the code is better depends entirely on whether the loop happened to converge before the clock ran out.
Most of the time it does. The pipeline earns its keep on those runs, and the round budget is a reasonable constraint. The problem is that the exit condition is the same whether the loop converged or gave up. The pipeline does not distinguish between "done" and "done trying," and without something watching for that difference, neither does anyone else
When it does not, the pipeline does not stop; it just stops trying.
Key takeaways
- Round budgets do not guarantee quality: Reaching a review round limit simply means the pipeline hit its allocated budget, not that the security issues or specifications have been successfully resolved.
- The risk of non-convergence: When different AI reviewers agree on a vulnerability but disagree on the exact phrasing or logic of the fix, the implementer agent can fail to satisfy all requirements simultaneously, leaving critical security bypasses unresolved.
- Difficulty with cross-functional invariants: AI agents struggle to resolve dependencies that require code in one function to remain consistent with security controls in another because they tend to revise one file at a time in isolation.
- Instrument loops to detect failures: Pipelines should actively track the number of findings; a flat or increasing count, or a specific finding repeated in consecutive rounds, serves as a clear non-convergence signal.
- Mandate human escalation: If the count of blocking findings does not strictly decrease between rounds, the pipeline must trigger human escalation rather than ship the code.
- Treat invariants as high-risk: Cross-functional dependencies do not surface well in flat lists of findings and should be flagged as specific convergence risks for automated reviewers.
