# Agents Already Gate the Merge. Nobody Agrees What Counts as Proof.

> Agents already decide which pull requests reach a human. What nobody has settled is what counts as evidence, and very little of it comes from the running product.

Source: https://qa.tech/blog/who-gates-the-merge · Published: 2026-10-30

---
At a growing number of engineering organisations, an agent decides whether a pull request reaches a human reviewer, and sometimes whether it needs one at all. That part is settled. Anthropic, Uber, the Kiro team at AWS and a handful of independent builders have all written up how their gates work.

What isn't settled is what the gate should look at.

Read the published systems side by side and a pattern shows up fast. Almost every one of them reads the diff, or reads test results that the submitting side produced. Very few of them open the product and use it. That gap is the subject of this post.

## The short version

An agent gates the merge at a growing number of companies, and the published designs agree on the routing: the change picks the tier, a missing or errored signal escalates rather than relaxes, and some paths always need a person. They disagree on the evidence. Anthropic's Claude Code team, Uber, Kiro, Gittifacts and Sphinx all gate on the diff or on test results the submitting side produced. Almost nothing in that evidence comes from the running product. The design rule worth copying is the one they share: the thing that checks the work is never the thing that produced it, and it starts with fresh context.

## The gate is already automated

Start with the Claude Code team at Anthropic, because they publish the most detail. Their [write-up on code review](https://claude.dev/blog/solving-code-review-at-anthropic/) covers that team's own flow rather than a company-wide policy, and inside it the rule is flat: "Every pull request gets a Claude review before a person looks." Not a suggestion. A step in the pipeline. "Most changes have been reviewed and fixed by Claude at least once before a human is asked to look."

Routing is automated too. "A classifier reads the change and picks its risk tier." Low tier covers docs, copy, internal-only features and additive changes behind a flag, and needs no human approval "once automated review and security checks are green". Medium tier, one person. High tier, two.

The line that matters for anyone designing this: "The change sets the tier, not the author." Seniority buys you nothing. And there are hard carve-outs: "Some paths always need a person, anything owners have listed as sensitive and any CI workflow file."

Context for why they built it: they have said before that Claude writes about 90 percent of Claude Code's own code.

That volume problem is the one engineering leaders describe to us in almost identical words. An engineering leader at a labour-marketplace company told us that "all of my teams are using AI for development. I think 80 plus percent of our code is AI generated." Then the consequence: "there was this one engineer who would review... 10 pull requests a day, which is like, come on, somebody help me out here. Right?"

On an AlphaSignal webinar in September, QA.tech co-founder Vilhelm von Ehrenheim drew the line where most teams draw it: "I think, as it stands now, I think you should have a human reviewer in the loop. The question is, just like, when agents are producing more and more code and more and more PRs, It takes a very long time for a human to kind of actually look through all of that code." He added that he knows "companies that have that kind of automatically merge low risk PRs, for example."

Uber went narrower and harder. Their [AI code cleanup programme](https://www.uber.com/us/en/blog/ai-code-cleanup/) uses a reviewer agent per property, four properties: no additions, preserves semantics, no intent drift, no agent artifacts. "Each property gets its own reviewer agent with a prompt focused entirely on checking that one thing." And the enforcement is absolute. "Any failure: the diff is abandoned before a human ever sees it".

The properties are written as rules about the change itself. "The cleanup should only remove or simplify code, never introduce code". "The program's observable behavior must remain identical after cleanup."

Kiro, at AWS, went the opposite direction on judgement. [Their post on how Kiro fixes itself](https://kiro.dev/blog/how-kiro-fixes-itself/) says "Models can propose changes, but we use deterministic code to accept or reject them," and that "The gate, which is plain code, decides whether the change survives." The page describes no model in the accept path. They describe "re-running the control every time" and keeping "a permanent regression suite". First week of running it: eight pull requests opened, "Seven merged". One example they give is a compaction retention score, a 0-to-1 measure of whether work can continue from a summary: "Retention went from 0.82 to 0.92, and it shipped."

The page also describes a human step that stays: "the pull request still goes through normal code review before it merges".

Devin's [on-call feature](https://devin.ai/blog/introducing-devin-oncall) adds a second agent "that runs asynchronously alongside Devin" whose job is to catch bad reasoning. It "examines recent reasoning, tool calls, and results for errors such as treating correlation as causation." Their example is sharp: "A recent deployment, for example, is a lead to investigate, not a causal explanation." That one gates a diagnosis, not a merge, but the shape is the same.

Then there is [Gittifacts](https://jovidecroock.com/blog/gittifacts), Jovi De Croock's design, which is the most explicit routing system of the lot. Three lanes. Lane 0 is auto-merge, when "everything is proven and nothing is risky." Lane 1 is quick review for "additive API, moderate size, some optional evidence missing." Lane 2 is full review for "breaking API, auth or migrations, new dependencies, scope drift, a fix without proof."

The lane is chosen by a pure function that "takes the evidence, the policy and the Plan and returns a lane with its reasons." Jovi calls it the classifier, and notes it has "no I/O or clock, so you can read why a Change landed where it did." A human approves the plan before any of this. Missing or errored signals can only push a Change into a stricter lane, never a looser one. On his own repo, generating the evidence "took 13 to 17 minutes per Change."

Different philosophies, same outcome: in each of these systems the gate runs without a person in it most of the time.

## What all this evidence is evidence of

Now line the inputs up.

Anthropic's classifier reads the change. Uber's four properties are four properties of the diff. Kiro's control is a script re-run against a saved scenario. Gittifacts routes on an evidence bundle. Sphinx, which we will get to, reads the diff and asks you about it.

The common ancestor is the diff, plus whatever the author's own CI reported.

Jovi states the dependency outright: "Auto-merge still trusts the author's test environment for parts of the evidence." The parts he lists are test runs, red/green, coverage and mutation. All four come from the same side of the table as the change.

So the gate asks: did the author's tests pass? It does not ask: does the thing work now?

Those are different questions. A diff can be clean, typed, covered, reviewed by three agents and semantically faithful to its stated intent, and the checkout flow can still be broken, because the breakage lives in an interaction between this change and four others that landed the same afternoon. Nothing in a diff-shaped gate is positioned to see that. We wrote more about the distinction in [how to verify coding agent work](https://qa.tech/blog/how-to-verify-coding-agent-work) and in [passing tests are not proof](https://qa.tech/blog/passing-tests-are-not-proof).

## Where an independent exercise of the product fits

QA.tech, an AI testing platform, is what we built for that slot. The limits first, because they decide whether the rest is any use to you: Chromium for web, native iOS and Android as separate surfaces, and no Playwright or test-code export, because the agent is the runner rather than a generator of something you run later.

On a pull request, an agent that did not write the code opens the running application and uses it. It has already crawled the app in a separate pass somebody started, and holds a knowledge graph of what the product contains. On the pull request it uses the diff to weigh risk, selects flows from that graph, and writes any the change needs that do not exist yet. Those generated tests persist as Drafts labelled `ephemeral` and run as part of the review, outside the scheduled suite. `ephemeral` is a label, not a lifespan: the tests are not discarded. It drives those flows like a person would. Then it returns one of three verdicts on the PR: pass, fail, or "unable to verify".

Scheduled regression is a separate decision. If enabled, the opt-in post-merge agent reviews the test cases surfaced during that PR's review and activates only the ones worth keeping, labelling them `regression` and `auto-added`. As the GitHub documentation puts it, "Promoting nothing is a normal outcome." Generating a test does not automatically add it to the scheduled suite; that requires deliberate promotion.

The third one is the one teams get wrong. An agent that can't reach the flow it was asked to check should say so, not guess, and "unable to verify" is a routing signal in the same way Gittifacts treats a missing input: it should push the change toward stricter review, never looser.

The risk-weighting is what buyers ask about first. A CTO at a healthcare software company described the behaviour he wanted before he described the product: "if a merge request gets pulled up, I would want this check to be risk adverse [sic]. ... for this level of change, you only need to check this flow. But this change actually is going to be huge. So we need to check 10 flows because it's not the same level of test."

That is the same instinct as Anthropic's tiers, applied to flows instead of approvers. The change sets how much product gets exercised. If you want the manual version of that logic first, our [pull request checklist](https://qa.tech/blog/pull-request-checklist) lays out what to check per change type, and [regression testing in pull requests](https://qa.tech/blog/regression-testing-in-pull-requests) covers the repeat-coverage side.

## Tests passing and being mergeable are not the same thing

There is direct evidence that green CI and a merge decision come apart. The headline number is the least useful part of it.

In a [March 2026 write-up](https://metr.org/notes/2026-03-10-many-swe-bench-passing-prs-would-not-be-merged-into-main/), METR reported on a study where maintainers from scikit-learn, the Sphinx documentation project and pytest blind-reviewed 296 AI-written patches that had already passed the SWE-bench automated grader. The finding: "roughly half of test-passing SWE-bench Verified PRs written by mid-2024 to mid/late-2025 agents would not be merged", and that the automated grader, normalised against the human baseline, runs "on average about 24.2 percentage points (standard error: 2.7) higher than the maintainer merge decision". Unnormalised the gap is 34.9 points.

The study's own control cuts the number down: about 68% of the human-written golden patches were merged, so a third of those were rejected too. The gap is real, and it sits on a baseline that was never 100 percent.

One more caveat. METR measured SWE-bench tasks against open-source maintainer judgement. It did not measure shipped product behaviour. Reading it as "CI is not proof your feature works" is an inference, not a measurement.

## The opposing bet: gate on human comprehension

Not everyone thinks the answer is more machine evidence. [Sphinx](https://github.com/parthh01/sphinx) makes the opposite wager, that the thing worth protecting is whether the person pushing the change understands it.

"Git pauses on the branches you protect." Then it "asks what the change does when things go wrong." Their example is a webhook handler that saves an event ID before calling charge(), and the question is what the next retry does when the charge fails. "Get it wrong and you learn why, then try again, usually with a new scenario." The design detail that gives it teeth: "A reviewer that never saw your chat reads the diff and asks what the change does when things go wrong." No shared context, so you cannot pass by gesturing at a conversation you had with a model an hour ago.

In fairness, there is an escape hatch, and they document it: "`sphinx yolo` lets pushes through, and every skip is logged."

The page describes a comprehension check on the diff. It doesn't describe exercising the running product. That is a design choice rather than an oversight, and the two approaches are not in competition. A person who can explain the retry path and an agent that actually triggers a failed charge in staging are answering different questions. Most teams need both answers.

## The one design principle worth copying from all of them

Cloudflare's [security audit skill](https://github.com/cloudflare/security-audit-skill) is at 47.5k stars as of 26 October 2026, and its central rule is one line: "The agent that checks a finding is never the agent that found it." They "give every unique candidate to a fresh verifier that tries to disprove it."

The Claude Code team lands in the same place by a different route. Their practice is to give Claude something it can run, like tests, a build or a screenshot to compare against the design, and then "ask it to show the output instead of saying it's done. Then a subagent reviews the diff with fresh context."

The checker is not the finder, and it starts with fresh context.

Every durable gate in this list has that property somewhere. Uber's reviewer agents each check exactly one thing and know nothing about the others. Sphinx's reviewer never saw your chat. Gittifacts' classifier is a pure function, and a missing or errored signal "can only raise the lane, never lower it." The pattern is independence, and the strictest form of independence available is an evaluator that did not write the code and judges it by using the product instead of by reading it.

## Where this leaves a team shipping this quarter

You do not get to opt out of this, because PR volume is already past what human review absorbs. So the practical question is narrow: when your gate says yes, what did it actually check?

Write down your tiers before you automate them. Let the change pick the tier, not the person. Keep a short list of paths that always need a human, and put your CI configuration on it. Treat a missing or errored signal as a reason to escalate. And at least once, ask what proportion of your merge evidence came from the same side of the table as the change. For most teams the answer is all of it.

That is fixable. It just is not fixed by adding a sixth agent that reads the diff.

## FAQ

**Are agents really blocking merges at real companies, or is this a demo?**

Both Uber and Anthropic have published how theirs works. Uber's write-up describes a diff being abandoned before any human sees it if any one of the four property reviewers fails it. Anthropic's write-up describes low-risk changes routed to zero required human approvals once automated checks are green. Kiro's gate is plain code that accepts or rejects automatically, though their pull requests still go through normal code review afterwards.

**What is the difference between evidence about a diff and evidence about the product?**

Diff evidence describes the change: size, scope, semantics, coverage, whether it drifted from its stated intent. Product evidence describes behaviour in the running application after the change lands. A diff can be completely clean while a flow is broken by an interaction the diff never touches.

**Does green CI mean a pull request is mergeable?**

Less often than people assume. METR found roughly half of test-passing SWE-bench patches from recent AI agents would not be merged by the relevant maintainers, with merge decisions about 24 percentage points below grader scores once normalised against the human baseline, and 34.9 points below unnormalised. Context matters though: about 68% of human-written control patches were merged, so a third of those were rejected too.

**Should we auto-merge low-risk pull requests?**

Some companies already do. The useful discipline is the one Anthropic describes: the change sets the tier, not the author, and some paths never qualify no matter how small the change looks. CI workflow files are a good first entry on that never-auto list.

**What does "unable to verify" mean as a gate result?**

It means the agent could not reach the flow it was asked to check, so it isn't claiming a pass or a fail. Treated correctly it escalates the change to stricter review. Treated badly it gets ignored, which is how teams end up trusting a gate that did not run.

**Can QA.tech test our native mobile app and our web app together?**

They are separate surfaces. Web testing runs on Chromium; native iOS and Android are handled separately rather than as one combined run.

**Does any of this remove the human reviewer?**

Not in any of the systems described here. Anthropic keeps one or two humans on medium and high tiers and a permanent list of sensitive paths. Kiro keeps normal code review. Gittifacts has a human approve the plan up front. What changes is which pull requests humans spend their attention on.