Test impact analysis (TIA) picks the subset of your existing tests that a code change can affect and runs only those, so CI skips the rest of the suite. It works by mapping code to tests, through coverage data, build dependencies or a model trained on past failures. For unit and integration tests in one codebase it can roughly halve test cost, as Meta's numbers below show. For end-to-end tests it gets much weaker, because the line from a changed file to a user flow is long and full of CSS, config and shared components the map can't follow.
So use it where it's strong, and select end-to-end checks by the user flows a change can reach.
Why test impact analysis pays off
On a given change, the large majority of tests tell you nothing. Google's 2017 study of its own continuous testing system, Taming Google-Scale Continuous Testing, counted 150 million test runs a day. Only 1.23% of the tests it analysed ever found a breakage, and for each code change passing test targets outnumbered failing ones 99 to 1.
Running everything every time doesn't add much signal. It does add flaky failures, queue time and cost, and the cost grows with every test someone adds and nobody deletes.
How test impact analysis works
There are three broad approaches.
Dependency and coverage maps. Record which code each test executes, then on each commit run the tests that touch changed code. Microsoft's Test Impact Analysis for Azure Pipelines is the textbook example: it "automatically selects only the subset of tests required to validate the code being committed." It's scoped to .NET managed code with tests and app on a single machine, and "for commits and scenarios that TIA can't understand, it falls back to running all tests."
Predictive selection from history. Train a model on which tests failed for which past changes. Meta's predictive test selection cut its testing infrastructure cost by a factor of two while still reporting over 95% of individual test failures and over 99.9% of faulty changes back to developers. Gradle's Develocity Predictive Test Selection does the same for Gradle and Maven builds and is open about the deal: it "trades testing comprehensiveness for faster feedback."
Language-model similarity. Newer tools read the change and the tests and score how related they are. CloudBees Smart Tests, from CloudBees, which acquired Launchable in August 2024, uses large language models "to understand your code changes and automatically identify the most relevant tests to run."
Where it breaks down: end-to-end tests
Every approach above learns from the link between changed code and tests, whether through coverage or failure history, and that link is loose for an end-to-end test of checkout.
An end-to-end test of checkout exercises a large share of your application. Gradle's docs see it the other way round: they say history-based selection matters most for "long-running tests such as integration tests and end-to-end tests where a large amount of code-under-test is typically executed during a test", because coverage-based tools struggle there. A coverage map says almost any change could affect them, so either everything runs or the selection is too coarse to help.
Then there's everything a dependency graph can't reason about. Microsoft's TIA, by design, gives up on markup: "if the code commit contains changes to HTML or CSS files, it can't reason about them and falls back to running all tests." Styles, feature flags, config and translations change what a user sees without changing which functions get called. Dependency upgrades to UI packages fall in the same gap, which is why our Dependabot auto-merge policy puts an end-to-end check in front of them.
An engineering lead at an AI-video SaaS company described the result in their own pipeline: "it's not a smart system, right? So it's basically just as the dependency graph … It could be some configuration, it could be even, I don't know, unit tests or something. So in that case, it's still marked as affected." Selection only pays off when it can tell changes apart, and a graph that flags configuration and unit tests as affected can't.
Selecting end-to-end tests by user flow
For end-to-end checks, start from which user flows this change can reach.
Our co-founder and CEO, Daniel Mauno Pettersson, starts from the component, because a team that changes one already knows which views it appears on, and those views are the flows to exercise. Edit the address form used in sign-up, checkout and account settings, and all three flows are in scope. Edit the profile-photo uploader, and only one is.
The selection process then looks like this.
- Read what the change is meant to do, from the PR description and linked ticket as well as the diff.
- Work out which user flows it can reach, through components, routes and the API calls those flows make.
- Run the existing end-to-end tests that cover those flows.
- Write tests for affected flows that nothing covers yet, and scale how many by risk.
Step 4 is the one classic TIA can't do. It can only choose from tests that already exist, and a brand-new feature's flow has none yet. Pre-writing them all doesn't scale. As our co-founder and Chief AI Officer Vilhelm von Ehrenheim put it, "you need to have hundreds of thousands of tests in order to test every nitty-gritty detail. And it would also be extremely brittle as something changes over time."
What he describes instead is generation scoped to the change: "the agent kind of looks at the diffs and the context it has about the application. And then it can run small, isolated tests just for those features. And then you get a little bit more testing upstream before you actually run larger regression suites."
Keep both
Daniel's split is the one we'd recommend: whatever is quick to run stays in your codebase and runs on every commit, with TIA trimming it if the suite is large. The slow end-to-end checks run per change, picked by blast radius and risk. A small scheduled regression suite covers the flows that must never break.
If you want the full argument for keeping that suite small, it's in a large regression suite is a liability. How deep each change should be tested is covered in risk-based testing for pull requests.
How QA.tech selects tests for a change
Selection is a judgement, and ours can miss a flow. When a review can't exercise the change, its verdict is "unable to verify", which never counts as a pass. Each pull request is read by QA.tech, an AI testing platform, which semantically matches the changes to existing test goals. Matched test cases run on the PR's preview, and new ones get written only where no existing goal covers a path, more of them when the change scores higher risk. That's dynamic PR testing in practice: end-to-end selection by flow, with generation for the gaps.
FAQ
What is test impact analysis? A technique that selects and runs only the tests affected by a code change, using coverage data, build dependencies or a model trained on past failures, instead of running the full test suite.
What is the difference between test impact analysis and predictive test selection? Classic test impact analysis uses a map of which code each test executes. Predictive test selection uses a model trained on which tests failed for past changes. Meta and Gradle both use the predictive approach.
Does test impact analysis work for end-to-end tests? Coverage-based TIA struggles with them: an end-to-end test executes a large share of the application, so almost every change looks relevant, and Microsoft's TIA falls back to running all tests on HTML or CSS changes. History-based tools like Gradle's claim better results here, but they can still only pick from tests that already exist.
What are the main test impact analysis tools? Microsoft Test Impact Analysis (Azure Pipelines, .NET managed code, coverage-based), Gradle Develocity Predictive Test Selection (Gradle and Maven builds, history-based) and CloudBees Smart Tests (LLM-based predictive test selection).
Is it safe to skip tests that test impact analysis doesn't select? For unit and integration tests, mostly, if you still run the full suite on a schedule to catch what selection misses. Meta's model still reports over 99.9% of faulty changes, but lets up to 5% of individual test failures through. For end-to-end checks, select by user flow instead.