The AI cost of shipping a feature is the sum of four stages: generating the code, reviewing it, verifying it in the running product, and watching it in production. Each stage has an AI bill and a human-time bill. Token spend on generation is the part that shows up as one clear line on an invoice, and we've found no public benchmark for what the other three should cost.
What does the AI cost of shipping a feature include?
A feature isn't shipped when the agent stops typing. Someone still has to read the change. Something has to prove it works in the running app, and then it has to get through its first weeks in production without paging anyone. Leave a stage out and the number you take to the budget meeting is wrong.
Engineering leaders have started budgeting this way. In our conversations with CTOs this autumn, two figures keep coming up side by side: a cost per pull request and a cost of verifying it. From there it's mostly about routing. Which model does which task? Does every change need a test run, a security review, a person in the loop?
Our article on DORA metrics in the AI era asks how much you should spend testing a change relative to what it cost to generate, and leaves the question open. The method here is to model one feature with your own figures. Why verification deserves a row of its own is argued in Can You Run a Software Factory Without a Verification Layer?. Here it's just a row with a cost.
Why token spend is only the visible part
Tokens get the attention because they're the line a finance team can see. Every sourced figure in this article was checked on 16 Nov 2026.
Claude Code's cost documentation puts the enterprise average at around $13 per developer per active day, or $150 to $250 per developer per month, with 90% of users staying under $30 per active day. That's a vendor's own average. Heavy agent use looks different.
In The Pragmatic Engineer's October survey of 27 workplaces, a two-person startup spends $3,000 per developer per month while a public company with 1,500 engineers caps each developer at $150. Size didn't predict the bill. And James Shore reports one CIO whose top users each spent $21,000 a month on tokens.
The price is moving too. LeadDev's argument is that a per-seat plan is a subsidy that may end, and that per-token rates cost roughly ten times more. Several companies in the Pragmatic Engineer survey are preparing for their bill to rise 5 to 10 times as subsidised subscriptions end, unless they optimise first. Vendors have already answered heavier agentic usage with price rises and usage limits, as LeadDev's budget coverage sets out. We hear the same worry on calls: that today's tokens are subsidised and the subsidy won't last.
Then there's what gets measured. LeadDev reports that 58% of respondents to its AI impact research say their organisation measures AI impact by token usage, and 57% say that's ineffective at gauging real value. Token maxing was fashionable for a while, and it was never going to work as a budget.
The three stages after generation are spread across other invoices and across people's calendars.
A cost model for one feature
Take one feature you shipped recently, list its pull requests, and fill in a row per stage.
| Stage | What you pay for | Who pays in time | How to estimate it |
|---|---|---|---|
| Generate | Model tokens or seat plans for coding agents | The engineer who writes the spec, steers the agent and reads what it produced | Token spend from your provider's usage export, tagged by PR or branch. Hours from the ticket |
| Review | Automated code review runs, charged again on each push in some setups | Human reviewers, often your most senior people | Review tool spend divided by PRs reviewed. Reviewers' hour counts, plus rounds and waiting time from your Git host |
| Verify in the running product | Preview environments, CI minutes, test agent runs or script execution, reruns | Whoever opens the preview and clicks around. Whoever triages a failed check | CI and test tooling bills divided by PRs. Minutes of manual checking per PR. Rerun count |
| Watch in production | Observability and error tracking, AI triage of alerts, rollback tooling | On-call engineers, support, the author pulled back in | A share of the observability bill. On-call hours and incidents traced to the feature in its first 30 days |
| Rework (applies to every row) | Every stage above, paid again | Everyone above, again | PRs sent back divided by PRs opened. Reverts and hotfixes per feature |
For each stage, add the AI and compute spend to the human hours multiplied by a loaded hourly rate. Count repeat rounds once, in the rework row. Then sum the rows.
Don't expect vendors to hand you unit prices for the middle rows. Anthropic's documentation for its automated code review says the feature "has no flat per-review price": cost scales with the size of the change and the depth of the review, and a review after every push multiplies it by the number of pushes. The same page tells you to run a small pilot and read your own average cost per review before a wide rollout. That advice holds for every row here. For the production row, our guide to monitoring tools for AI-first engineering teams compares five of them on setup and pricing.
What is the testing-to-coding cost ratio?
It's what you spend verifying a change divided by what you spent generating it. We don't know of a clean public benchmark for it. We went looking and found five data points from five different setups, each measuring something slightly different.
| Source | What it reports | What you can take from it |
|---|---|---|
| Stripe, Feb 2026 | Its coding agents get at most two rounds of CI per run, because "CI runs cost tokens, compute, and time" | Verification is costly enough at scale that a large engineering org rations it |
| Anthropic, Oct 2026 | "Pulling a senior engineer out of deep work to catch a routine mistake costs far more than an automated review." | At Anthropic, a senior reviewer's time is the expensive side of the ratio |
| ChaCo study, Jan 2026 | Generating PR-specific unit tests cost an average of $0.11 per PR across 145 PRs from three open-source Python projects | Unit-level test generation is cheap. It says nothing about browser-level checks |
| One practitioner writing on DEV about Playwright agents, May 2026 | "I ran through 78% of my 5-hour usage on the planner alone." | An agent driving a browser can eat a subscription window before a single test runs. One person's account |
| Gumloop, Oct 2026 | After the team cut its agent instructions by 44%, the median cloud agent run cost 20% less the following week | Context size is a cost lever. Vendor self-reported |
None of these is a ratio, and we haven't seen a general one we'd trust. So treat it as a question you put to your own data, three ways:
- Tokens only: verification tokens divided by generation tokens.
- All AI and compute: add CI minutes, preview environments and test tooling.
- Fully loaded: add the human hours on both sides.
The third is the one to argue about. And it shouldn't be a single figure for the whole company. A copy fix and a change to checkout don't deserve the same spend, which is the case for setting verification depth by risk. Vilhelm von Ehrenheim, our co-founder and Chief AI Officer, describes the same trade from the product side: "There's always this balance between risk and cost, and intelligence and speed."
Verification spend has always been a dial. A QA team of any size could have tested more with more people. Now the dial is priced in tokens and turns per pull request.
A worked example, with illustrative numbers
This section is an illustration. Every figure in it is made up to show the arithmetic. None is a benchmark, a customer number or a price for any product.
Say one engineer ships a mid-sized feature, working with a coding agent. The team hasn't bought a testing tool: verification here is CI minutes, preview hosting and someone clicking around the preview. A loaded engineering hour costs $100. A quarter of the work fails a check and goes round again.
| Stage | AI and compute | Human time | Human cost | Stage total |
|---|---|---|---|---|
| Generate | $20.00 | 3 h | $300 | $320.00 |
| Review | $5.00 | 2 h | $200 | $205.00 |
| Verify in the running product (CI, previews, manual checks) | $10.00 | 1 h | $100 | $110.00 |
| Watch in production | $5.00 | 0.5 h | $50 | $55.00 |
| Rework (a quarter of the work repeats the first three stages) | $8.75 | 1.5 h | $150 | $158.75 |
| Total | $48.75 | 8 h | $800 | $848.75 |
Generation tokens are $20 of $848.75, a little over 2%. All AI and compute together come to under 6%. The rest is people.
The testing-to-coding ratio depends on which version you compute. With these made-up figures, compute alone is $10 against $20, a ratio of 0.5. Fully loaded it's $110 against $320, about 0.34. Neither is a target.
Now look at where a cut pays. Halving the entire AI bill saves about $24. Taking one hour of human time out of the feature saves $100. The rework row alone costs more than three times the AI bill.
In an example shaped like this one, the useful question about any verification spend, a test agent included, is whether it takes hours out of review and stops rework rounds. Add its cost to the verify row at whatever your own pilot shows, then watch the time column. Your shape may differ. A team running fleets of agents will see a much larger AI share, and the same table will show it.
Where the costs hide
Rework
Rework is every earlier stage paid for again. "If you produce twice as many features, but double your maintenance costs, your team's overall productivity spikes, then drops like a rock," James Shore writes. Faros reports monthly incidents up 125.4% while incidents per PR rose 14.5% in its Q3 2026 report. Each change is only slightly riskier, and there are far more changes. That's one vendor's dataset with no baseline stated, so read it as a direction. Count PRs sent back, reverts and hotfixes per feature.
Review time
Bigger changes are landing on the same reviewers. The same Faros post reports average PR size up 71.8% and PRs merged with no review up 76.3%, and its reading is that long queues can push teams to skip review to keep work moving. In Faros's account the checking then lands downstream in QA, where it reports time in QA up 300.6%, the largest deterioration among its flow metrics.
More spend doesn't buy more merged work by itself, either. At one public company in the Pragmatic Engineer survey, PR frequency stayed flat while PR size tripled, with most backend engineers spending around $1,500 a month and team leads around $2,500, against a soft $1,000 cap.
Review rounds and waiting time are the easiest hidden costs to measure. Your Git host already has the timestamps.
False failures
A red check that turns out to be nothing still pulls a person out of their work. Then it does something worse: it teaches the team that red might mean nothing. Track failed checks that led to no code change as a share of all failed checks. If that share is high, you're paying for verification and for the distrust it creates.
Flaky reruns
Rerunning is the cheapest fix for a flaky check. It hides well, because the cost is smeared across CI minutes, tokens and waiting. Count reruns per merged PR. If agents react to each failure, count what those reactions cost in tokens too.
What to cut first
Cut in this order.
- Verification of changes that can't break anything a user sees. Docs, CI config, build tooling. A triage step that skips them costs almost nothing, provided it follows skip rules like the ones in risk-based testing for pull requests.
- Extra rounds. Cap CI and review rounds per agent run, and think twice before reviewing every push.
- The biggest model on every task. LeadDev cites October figures from Artificial Analysis in which one model cost about $0.07 a task and another $0.55, at intelligence scores of 38 and 42. A frontier model on work a rule could do is a very big hammer for a very small nail.
- Context. It's tempting to cram more into the prompt and hope for the best. That makes runs slower and dearer without making them better. Browser agents are the worst case, since the page goes back into the context at every step, which is where the tokens go when an agent drives a browser.
- AI where a script will do. A scripted check that already runs reliably costs a few CI minutes per run and gives the same answer every time, so leave it running. Asking a model to redo what a script already proves is the dearest way to get the same answer.
Cut last: human judgement on high-risk changes, and verification of the flows that carry revenue. A staff engineer told The Pragmatic Engineer, "We still do human PR reviews, and they catch a lot of stuff that agents and agentic review misses." Anthropic's rule for its own review savings is the test to copy: "Revert and incident rates are the guardrail: if they rise, we haven't saved human time, we've only moved it."
What a verification platform changes in the cost model
A verification platform covers one row of the cost model: verifying the change in the running product. It won't shrink your generation bill. It doesn't give opinions on code quality, and it isn't production monitoring. If you already have scripted end-to-end tests that run reliably, they should stay.
QA.tech, an AI testing platform, tests the running app on a pull request's preview and closes each review with pass, fail or unable to verify. Behind the verdict is a run, linked from the PR, with screenshots, logs and a trace to read, and sometimes a recording. A reviewer can go through that before deciding whether to open the preview and click around.
Some of the cost levers above are built into PR testing. The depth of a run is set by risk: how many flows a change can reach, how central they are and what's already covered, so a typo fix and a payment change don't get the same run. A review reuses existing test cases before it writes new ones. On GitHub, PR triage (on by default for new projects) skips changes that can't be tested end to end, such as docs-only or CI-only pull requests, and the check says it was skipped and why. The pricing page also lists spending limits and budget alerts on Starter, Growth and Enterprise.
Starter is $89 a month for up to 5 users and 2 applications, and there's a free trial for one application. Growth and Enterprise are priced to volume. So if the question on your list is where you can buy a subscription for an AI-driven end-to-end testing service without booking a call first, Starter is the self-serve route.
What any of this does to your review hours and rework rounds is for a pilot to show. Fill in the table for a few weeks before and again after. You can also build this row yourself with a coding agent and a browser tool, and Build or Buy Your QA Agents? has a two-week pilot for comparing the two routes. The case for an engineering leader, without the arithmetic, is on our page about AI testing for CTOs.
How to fill in the model for your own team
- Pick five features shipped last month, of mixed size and risk.
- Tag AI spend by PR or branch, using your provider's usage export.
- Pull review timestamps from your Git host, then count review rounds, reruns and PRs sent back.
- Ask each author and reviewer for a rough hour count per stage.
- Price the AI spend twice, once at what you pay now and once at list rates.
- Compute the three ratios per feature, split by risk, and repeat next quarter.
FAQ
What does it cost to ship a feature with AI coding agents?
There's no single figure, and the spread between teams is wide. For generation alone, Claude Code's docs put the enterprise average at $150 to $250 per developer per month, and The Pragmatic Engineer found one startup spending $3,000 per developer per month. Review, verification, monitoring and the human hours around them come on top. Only your own data can price those.
What is the testing-to-coding cost ratio?
It's the spend on verifying a change divided by the spend on generating it. We haven't found a public benchmark for it. Compute it three ways (tokens only, all AI and compute, fully loaded with human time) and compare it across risk levels inside your own team.
How much does it cost to have coding agents write and maintain tests themselves?
For unit tests, the ChaCo study measured an average of $0.11 per PR on three open-source Python projects. Browser-level tests cost more because the agent has to look at the page at every step, and one practitioner writing on DEV used 78% of a five-hour subscription window on the planning step alone. We haven't seen a published figure for the maintenance that follows.
Will AI coding costs rise when the subsidies end?
Possibly. LeadDev puts per-token rates at roughly ten times a per-seat plan, and several companies in The Pragmatic Engineer's survey are preparing for a 5 to 10 times increase unless they optimise. The same survey found spend has roughly halved since the summer peak at companies on API plans, so the direction isn't settled. Budget for both cases.
How do you measure ROI for improving QA with AI?
Measure what changes in the time column: review hours per PR, rework rounds, reverts, and incidents in a feature's first 30 days. Hours saved alone isn't a return, for the reasons set out in our piece on DORA metrics in the AI era. Comparing against manual QA is another calculation, the one in the ROI of intelligent testing.
What does skipping verification cost?
It moves the cost to production. Faros reports monthly incidents up 125.4% and PRs merged with no review up 76.3% in its Q3 2026 report. Those are one vendor's figures with no stated baseline, and Faros doesn't say one causes the other: in the same report, incidents per PR rose 14.5%, so most of the increase is volume. The mechanism is still simple. A defect nobody caught on the pull request gets paid for later, by on-call engineers and by customers.
