Contents
Agentic testing is software testing carried out by an AI agent that works from a goal rather than a script. You describe what should be true ("a user can reset their password and log in with the new one"). The agent looks at the live application, decides what to do next, does it, checks the result, and repeats until it can say whether the goal was met. Nobody writes steps and nobody maintains selectors, because there aren't any. When the interface changes, the agent works out the new path. What you do maintain is different: the goals themselves, the test data and accounts the agent uses, and the constraints you put on it.
"Agentic QA", "agentic AI testing" and "agentic test automation" all describe the same thing: the system has agency over how it reaches the goal. The term is newer than the idea. Open-source substrates such as Browser Use, Stagehand and Skyvern, and the computer-use models underneath them, made goal-driven browser agents buildable in 2024 and 2025; the testing platforms in this category are opinionated products on top of that layer.
What a goal actually looks like
A goal is closer to an acceptance criterion than to a test case.
As a logged-out user with an existing account, reset the password from the login page, follow the emailed link, set a new password, and confirm you can log in with it and not with the old one.
That's the whole test. What it deliberately doesn't say is which button to press, what the reset page is called, or where the link sits in the email, because those are the things you'd otherwise have to update the next time design touches that flow.
Agentic versus the things it gets confused with
The category is young enough that vendors stretch the word, usually onto something they already shipped. Here is what sits near agentic testing and how each differs.
Scripted automation (Playwright, Cypress, Selenium). A person or a code generator writes explicit steps tied to selectors. The test does exactly what it was told, every time, until the UI changes and it can't.
AI-assisted automation. A model writes the script for you, from a prompt or a recording. Authoring is faster. The artefact that runs in CI is still a script, with everything that implies for maintenance. Claude with the Playwright MCP, Cursor and Copilot all live here. Useful, and not agentic in the sense above, because at runtime nothing is reasoning.
Self-healing automation. A script with a repair mechanism: when a selector breaks, the tool guesses which element you meant and patches it. There's a small amount of runtime reasoning, aimed at keeping a script alive. It is agentic about one thing, finding the button, and scripted about everything else. We've argued elsewhere that the guess is the problem.
Autonomous testing. Often used interchangeably with agentic, and the overlap is large, but the emphasis differs. Agentic describes how the test runs (an agent reasons through it). Autonomous describes how much a human is involved (from "writes every goal and reads every verdict" to "sets policy and audits"). A tool can be agentic and still need a lot of you – what is autonomous testing works through the levels.
| Unit of authorship | Reasoning at runtime | Breaks when | Your job | |
|---|---|---|---|---|
| Scripted | Steps and selectors | None | The UI changes | Write and repair the script |
| AI-assisted | Steps and selectors, drafted by a model | None | The UI changes | Review the generated script, then repair it |
| Self-healing | Steps and selectors | Element lookup only | The heal picks the wrong element and the test goes green while checking nothing | Approve or reject the tool's guesses |
| Agentic | A goal | Plan, act and verify, every step | The goal is ambiguous, or the agent's judgement is wrong in either direction | Write goals, judge verdicts, fix the product |
There's one test that settles it, and it costs more to run than anything else on this page. Take the test you wrote, throw away the current UI, and rebuild it from scratch with the same behaviour: new markup, new component library, new labels, same product. A script has to be rewritten, because the script was a binding to the old UI's names and structure. An agentic test should still pass, because what you wrote down was the outcome. Then check the harder half: after the rebuild, does it still fail the things that should fail? An agent that has lost its footing tends to rationalise a pass. If the tool has a stored map of the old app, that map is a hint it can discard, not a dependency it needs.
How the agentic loop works
Every agentic testing system runs some version of the same loop. The differences between vendors are in how well each step is done, and in what the system remembers between runs.
Perceive. The agent takes in the current state of the application: a screenshot, and on web the page structure behind it, plus network activity and console output. Better systems read more than one of these. A request that failed behind a spinner exists only in the network log. Two buttons that look identical but do different things are told apart by the markup around them. An agent reading pixels alone misses both, and an agent reading only the DOM misses what a user actually sees: which element is prominent, what an icon means, whether the page looks broken.
Plan. Given the goal and what it can see, the agent decides the next action. An agent that plans badly wanders, retries, or invents work. In a trace it looks like an agent re-dismissing a cookie banner it already dismissed, then hunting for a password field in account settings when the field lives elsewhere, then running out of budget. Nothing crashed, and it never reached the thing you asked for. A good system constrains the plan: it knows the app's structure from a prior crawl, it has a budget for steps, and it knows when to stop and report rather than keep trying.
Act. Click, type, scroll, upload, switch tab, read a one-time code out of a test inbox. In a browser or a mobile emulator, driving your preview or staging environment the way a user would. Two details decide whether this step is reliable. The first is grounding: how "click the submit button" becomes an action on one specific element. Every approach has a failure mode. Resolve purely by where things are on the screen and the agent gets confused when two elements look alike or one moves; resolve purely by the markup and the agent misses what a user would see. When we built ours we found exactly that: a vision-only model fixated on the position of the target element, a text-only model couldn't tell a prominent button from a buried one, and combining the screenshot with the page's HTML was what made the same action recognisable after the page changed. Ask any vendor which signals their agent uses to identify an element and what happens when two candidates match. The second detail is settling: knowing that a request finished and the page stopped moving before the agent looks again. An agent that acts on a half-rendered page produces exactly the flakiness you adopted it to escape.
Verify. Look again, and judge. Did the outcome match the goal? A script asserts what its author anticipated. An agent evaluates what actually happened, which means it can flag things nobody thought to assert: a price in the wrong currency, an image that didn't load, an error logged behind a success toast. The trade is that the judge is itself a model. A scripted assertion is only wrong in one direction, in that it misses what its author didn't anticipate. A judged verdict can also be wrong the other way and call something broken that isn't, so the useful question to ask a vendor is what its false-positive rate looks like on a suite that is genuinely green. Where a judgement call isn't good enough, pin an explicit check inside the goal, for instance that a specific request returned a specific status and body.
Remember. A system with memory keeps a map of the application: its pages, the actions available on each, how features connect. A run reads the map, uses it to skip exploration it has already paid for, and writes back what it learned. A cold run on a new app spends most of its steps finding out where things are; a warm run on a known app spends them on the flow you asked about. Steps are what the work actually costs, whatever the invoice is denominated in, so a per-execution price moves the step count into the vendor's margin rather than off your bill. When you evaluate, ask for the step count of run one and run ten on the same goal. That ratio is the measurable version of any memory claim. The map can also go stale, the way any cache does, so ask what happens when it's wrong: the right answer is that the remembered path fails its check against the live page and the agent falls back to reasoning.
What makes a tool genuinely agentic
If you're evaluating vendors, these are the properties that separate the label from the thing.
- Goals, not steps, are the unit of authorship. If the "test" you write contains selectors, coordinates or step-by-step instructions the tool will replay verbatim, it isn't agentic.
- The path is a hypothesis, not a contract. Memory should make the agent faster at guessing where things are; it should never make it insist. Ask what happens when a button is renamed and moved. The right answer is that the remembered path fails its check against the live page, the agent falls back to reasoning, and the run passes anyway. A wrong answer involves a repair step, a quarantine, or a human.
- Verification is a judgement, not a scripted assertion. Ask the tool to catch something you didn't tell it to look for. Show it a page with a console error and see if the verdict mentions it.
- The run is bounded and the verdict is auditable. There should be a step budget, a stop condition that fires before the budget does, and a per-step record: screenshot, the reasoning that chose the action, the network and console output at that moment. The test is whether an engineer who didn't write the goal can open a red run and decide in two minutes whether it's a bug or the agent being wrong.
- It remembers, measurably. The run-one-to-run-ten step ratio above.
Where agentic testing wins, and where scripts still do
It wins on change. Products where the UI moves weekly, design-system migrations, front-ends being rewritten by AI coding tools at a pace no scripted suite can follow. A team merging 30 AI-assisted pull requests a week is generating UI churn faster than one person can reconcile a Playwright suite against it, and the suite loses: it gets skipped, then quarantined, then deleted.
The long tail finally gets covered. Most teams script an order of magnitude fewer flows than they ship, because only the top handful were ever worth the authoring cost. When a test is a sentence, that calculation changes, and the flows that only break for the customers who never file tickets start getting checked.
Scripts win on determinism. An agent is a system wrapped around a statistical model. Where a flow has one sensible path, runs look stable in practice; where there are two, you'll see both. If a check must be byte-identical every run, write a script. Snapshot tests, contract tests and anything asserting on an exact numeric output belong in code, and they should stay there after you adopt an agent.
Scripts win on raw speed per step. Reasoning is slower than replaying, and it will stay slower. The comparison that matters is wall-clock time to a trustworthy verdict, which includes the re-run you do because you didn't believe the first red.
Neither wins on the oracle problem. If the goal is wrong because the product spec was wrong, both the script and the agent will happily confirm the wrong behaviour. Goals have to come from what the product should do, which is tickets, acceptance criteria and a human's understanding. Agentic tools that read the issue tracker help; none of them replace the person who knows what "correct" means.
What it costs, what it can't do, and how to start
What it costs. Agentic runs are billed by work done, not by test count, so the number to model is steps per goal, times goals per week, times how often you run them. The variables that move it are memory (a warm app costs a fraction of a cold one) and scope (a run limited to the flows a diff touches costs far less than a full suite). Expect the first quarter to cost more than the steady state, because you're running the scripted suite alongside while you learn which flows to move.
What it can't do yet. Anything that happens faster than it looks. An agent samples the screen roughly once a second, so a flicker, a jank or a transition shorter than that can go unnoticed, and the recording is where you'd catch it. Third-party auth screens that actively fight automation, and anything behind a CAPTCHA. Flows that need a real payment instrument rather than a sandbox card. Judgement about visual design rather than behaviour. And anything where the correct answer depends on data the agent can't see, which is why test accounts with known state matter more here than they did with scripts.
Where the data goes. An agent driving your staging environment sends page content and screenshots to a model provider. Before a trial, ask what leaves your environment, where it's stored, for how long, and whether it trains anything. For QA.tech: SOC 2 Type 2, data at rest in the EU, model providers that don't retain or train on prompts, and terms that require synthetic test data.
How to start. Pick one flow that breaks often and that you've never scripted, write it as a goal, and run it against staging twice a day for a week. You're not measuring whether it passes. You're measuring what share of the reds were real, because that's the number that decides whether a red gets investigated or re-run. If it holds, wire it to pull requests for one service and leave the scripted suite exactly where it is.
Reproducibility policy. Decide it before you gate anything. A sensible default: re-run a red once automatically, block the merge only on two consecutive fails for the same goal, and quarantine a goal whose flake rate over 30 runs is above whatever your team already tolerates from its Playwright suite. Publish the numbers to the developers you're asking to trust the verdicts.
Blast radius. An agent with a browser on staging can send real emails, fire webhooks, hit third-party sandboxes and create records. The constraints you give it should be explicit: which hosts it may reach, which actions it must never take (delete an account, submit a real payment), and standing rules it applies on every run. In QA.tech those live as agent rules and network access controls; whatever the vendor calls them, ask to see where they're set before the agent gets credentials.
Where goals live. Ask whether goals are exportable as text so you can keep them in git and review changes like any other artefact. In QA.tech, test definitions with their steps export through the API and CLI. How the agent gets logged in is part of the same conversation: seeded accounts on a non-production environment, a mailbox for one-time codes, and a session it can reuse rather than re-doing SSO on every run.
Where it's being used
On the pull request. The agent reads the diff, decides which flows are affected, runs them against the preview environment and posts a verdict before merge. This is where the feedback is worth the most, because it arrives while the author still has the change in their head and before anything downstream depends on it. The scoped-to-the-change version has its own write-up: dynamic testing.
As a small regression suite. The flows that must never break, run on a schedule against production or staging, written as goals so they survive redesigns. Kept deliberately short; what regression testing is for makes the case.
Exploratory passes. The goal is still there, it's just broader: find things that look broken in this area. Expect lower precision and a triage cost, and expect it to surface the class of defect nobody wrote a ticket for, like a modal that traps focus or a list that renders empty for accounts with one item.
Native mobile. iOS and Android apps with the same goals as the web, and no per-platform scripts. On mobile the agent works entirely from what's on the screen, which is why custom-drawn and canvas-rendered UIs that defeat selector-based tools are fine here. The cost is that an icon with no label means nothing until someone tells the agent what it means, and that's a rule you write once.
Agentic testing at QA.tech
QA.tech is an agentic testing platform: the test is a goal in plain language, the agent maps your application into a knowledge graph that every run reads from and adds to, drives a real browser or mobile emulator, judges the outcome on every run and leaves screenshots, video, logs and its own reasoning as evidence. It perceives the way a user does, predominantly from the screen, with the DOM as a supporting signal on web and nothing but the screen on mobile; icons it can't read from context you teach it once, as a rule. It looks at the screen about once a second, so anything faster than that is for the recording, not the agent. Runs are bounded (a test case is capped at around 50 agent steps and stops when it runs out) and parallel executions are capped per environment by default, configurable, so the agents don't overload your staging. It posts verdicts to GitHub and GitLab pull requests, runs from CI, the CLI and the REST API, and exposes an MCP server so Claude Code, Cursor and Codex can run and inspect tests from the editor. It doesn't need your source code to run a test, though the pull-request integration reads the diff to decide what to re-test, and it doesn't export scripts, because there are none.
The rest of the picture, including the questions to put to any vendor in this category, is in the AI QA testing guide. The side-by-side with scripted suites is in agentic testing vs traditional test automation.