# Risk-Based Testing for Pull Requests: Skip, Test or Escalate

> Risk-based testing applied per pull request: which changes to skip, which to test lightly, and which deserve a full run and two reviewers.

Source: https://qa.tech/blog/risk-based-testing-pull-requests · Published: 2026-11-02

---
Risk-based testing means spending test effort where a failure is most likely and would hurt most. For years teams applied it once per release, in a test plan. Now that [coding agents open pull requests](https://qa.tech/blog/agent-proof-pull-requests) alongside people, the same idea has to run on every PR, and it comes down to three decisions: whether to test the change at all, how deep to go, and who has to sign off before it merges.

Testing every PR the same way gets both ends wrong. You pay for full runs on README edits, and the change that quietly touches checkout gets the same shallow pass as everything else.

## What is risk-based testing?

Risk-based testing in software testing is a test approach that sets what to test, how deeply and in what order by how likely a failure is and how much it would hurt. The [ISTQB glossary](https://glossary.istqb.org/en_US/term/risk-based-testing) defines it as "a test approach in which the management, selection, prioritization, and use of test activities and resources are based on corresponding risk types and risk levels." A [risk level](https://glossary.istqb.org/en_US/term/risk-level), in the same glossary, is defined by two things: impact and likelihood.

Impact is what breaks for the user and what it costs the business. Likelihood is how probable it is that this change breaks something. The classic version scored features in a spreadsheet once per release. Per-PR risk scoring does the same thing on every change as it arrives.

## Why it matters more per pull request

Volume is the reason. An adtech engineering lead told us his team went from 15 PRs per release to 150–200, and that "most of that volume has come from not necessarily a functional change to the application."

Some of those PRs can't break anything a user sees, and some can break everything. Treating them identically means you either spend far too much testing them or far too little on the ones that matter.

[Daniel Mauno Pettersson](https://www.linkedin.com/in/danielmaunopettersson/), our co-founder and CEO, asks customers the cost question directly: "are there PRs where we are running a lot of analysis, where you already know that we shouldn't be testing it in the first place? That is usually a big time token saver."

## Decision 1: should this change be tested at all?

Call this triage. Some changes have no behaviour a test could exercise: docs, READMEs, CI config, infrastructure manifests, build tooling. Running an end-to-end check on them produces cost and noise and nothing else.

Triage rules key off changed files, branches and who opened the PR. Skip if every changed file is under `docs/` or `.github/`. Skip PRs opened by a bot you don't need tested. Keep everything that touches the UI or the backend a user flow calls.

Two rules keep triage safe. A file-based skip should apply only when every changed file matches it, so a PR that mixes a README edit with a product change still gets tested. And there has to be an override, a comment or label that forces a full run when someone disagrees with the skip.

Dependency bumps are a category customers regularly ask us to skip, and the one where a blanket skip is riskiest. We cover that separately in [Dependabot auto-merge](https://qa.tech/blog/dependabot-auto-merge).

## Decision 2: how deep should the testing go?

This is where likelihood and impact come back. [Vilhelm von Ehrenheim](https://www.linkedin.com/in/vilhelm-von-ehrenheim/), our co-founder and Chief AI Officer, described the impact side at a CTO Craft session in September: "maybe there's just a small documentation change or something that has very low impact. Then the risk is obviously very low … But you might also have something that touches … the payment flows, or something that is revenue critical to your business, then you should be much more thorough."

Likelihood is harder, because the size of a diff is a bad proxy for it. Bumping your UI framework is a one-line change in a manifest that reaches every screen. A large refactor of an internal job can reach none of them.

The signals that work better:

- **Blast radius.** How many user flows can this change reach? A shared component or a framework version reaches all of them. A new settings page reaches one.
- **Flow importance.** Sign-up, payment, the flows that ruin your week if they break. These get more depth whatever the diff looks like.
- **Existing coverage.** If nothing tests the affected flow today, the change is riskier than its size suggests.
- **Churn.** In [Taming Google-Scale Continuous Testing](https://research.google/pubs/taming-google-scale-continuous-testing/), Google found that code recently modified by more than three developers breaks more often.

Put those together and you get a tier, and the tier decides how many flows get exercised and how deeply. Picking which existing tests cover those flows is its own problem, covered in [test impact analysis](https://qa.tech/blog/test-impact-analysis).

## Decision 3: who signs off?

The Claude Code team publishes its version of this. In [How the Claude Code team does code review](https://claude.dev/blog/solving-code-review-at-anthropic/), a classifier sets one of three tiers. Docs, copy, internal-only features and additive changes behind a flag need no human approval once automated review and security checks are green. Ordinary product changes need one person. The riskiest paths need two. And "the change sets the tier, not the author."

[Ona](https://ona.com/stories/auto-approving-low-risk-prs) went further on the approval side and auto-approves PRs that meet six low-risk criteria. Median time to first approval fell from 2 hours 49 minutes to 3.8 minutes. Engineers can't classify their own change as low-risk, and a human always clicks merge. Vilhelm sees the same pattern elsewhere: "I know companies that have that kind of automatically merge low risk PRs, for example."

In both setups, the person or agent who wrote the change doesn't get to decide how risky it is.

## A risk tier table for pull requests

| Tier | Typical changes | Testing | Sign-off |
|---|---|---|---|
| Skip | Docs, READMEs, CI config, infra manifests | None, with the skip reason recorded on the PR | Automated checks |
| Low | Copy, isolated new screens, changes behind a feature flag | The directly affected flow, end to end | Automated checks, optional reviewer |
| Medium | Ordinary product changes | Affected flows plus their neighbours | One reviewer |
| High | Payments, auth, shared components, framework or runtime major versions | Every critical flow the change can reach, in depth | Two reviewers |

Your categories will differ. Keep the shape: the tier comes from what the change touches, and every tier above "skip" is tested against a running build.

## What goes wrong

Misclassification is the first failure. A one-line change to a shared button component looks low-risk and lands in every view. Score blast radius from every place the code is used, which puts that button change in the high tier.

The second failure is invisible skips. If skips and tiers aren't visible on the PR, nobody notices when the rules drift. Record the reason on every skipped check and look at the skip rate once a month. If the skip rate keeps climbing, check the rules before assuming the code got safer.

## How QA.tech handles triage and risk

No set of skip rules gets every call right, so every skip has to be visible and reversible. QA.tech, an AI testing platform with GitHub PR integration, runs [PR triage](https://docs.qa.tech/pr-testing/pr-triage) before each review. On GitHub it's on by default for new projects, and it skips untestable changes such as docs, CI and build tooling using a rules prompt you can edit. You add your own skip rules in plain language, such as "skip if every file is under docs/" or "skip pull requests whose head branch starts with docs/". The rules see the PR title, the first 2,000 characters of the description, the base and head branches, who opened it, its deployments and the changed file paths. They never see the diff. A PR that changes more than 40 files is always reviewed. A skipped PR gets a check with the reason, and commenting `@qa.tech` runs the review anyway.

PRs that pass triage get a [dynamic PR testing](https://qa.tech/product/pr-testing) review weighted by blast radius, how central the affected flows are, and how much coverage already exists. The agent assigns a low, medium or high risk level and writes test cases only for uncovered paths, with their number and depth set by that level.

## FAQ

**What is risk-based testing?**
Risk-based testing is a test approach where you decide what to test, how much and in what order based on risk: how likely a failure is and how much it would hurt. ISTQB defines risk level by impact and likelihood.

**What is an example of risk-based testing?**
A team skips end-to-end tests on a README change, runs one flow for a new settings screen, and runs every payment flow plus a second reviewer for a change to the checkout component.

**How do you assess the risk of a pull request?**
Look at blast radius (how many user flows it can reach), how important those flows are, how much test coverage they already have, and how much churn the code has seen. Diff size alone is a poor signal.

**Should you skip testing low-risk pull requests?**
Skip the ones with nothing to test, like docs and CI config. Low-risk product changes still get a light end-to-end check on the flow they touch.

**Can low-risk pull requests merge automatically?**
Some teams do it. The Claude Code team requires no human approval for its lowest tier once automated review and security checks pass. Ona auto-approves low-risk PRs but still has a human merge. In both, the author can't set the tier.

**How is risk-based testing different from regression testing?**
Regression testing re-runs existing tests to check that old behaviour still works. Risk-based testing decides how much testing each change gets. They work together: a small regression suite for critical paths, and risk-scaled testing on every PR.