Ai·

Harness Engineering: The Sensor the Agent Harness Is Missing Is End-to-End Behaviour

Harness engineering splits the harness into guides and sensors. Nearly every sensor on the standard list reads code. What a behaviour sensor looks like, from a bug the agent was told about twice.

Vilhelm von Ehrenheim

Co-founder & Chief AI Officer

In April, Birgitta Böckeler published the clearest description I've read of what a coding-agent harness is. Her harness-engineering article on martinfowler.com splits the harness into guides, which steer the agent before it acts, and sensors, which observe after it acts so it can correct itself. Then it lists the sensors people actually run: custom linters, eslint, semgrep, type checkers, fast test suites, ArchUnit-style structural tests, dependency rules, coverage, mutation testing, dead-code detection, AI code review, human review, plus a couple of production-side ones like log anomaly checks.

Read that list again and notice what nearly every item has in common. They read the code. The few that don't watch production after the fact; nothing automated on the list opens the feature before merge. The feedback she describes for application behaviour is checking that the AI-generated test suite is green with decent coverage, then manual testing on top, and she adds that "this approach puts a lot of faith into the AI-generated tests, that's not good enough yet". Her February memo said it more directly about OpenAI's harness write-up: "What I am missing in the write-up is verification of functionality and behaviour".

I think that's the gap, and it's where every harness I've looked at this year sits. The short form first, then the incident that convinced me.

A harness without a behaviour sensor is an open loop: nothing between the ticket and the merge checks whether the running product does what the ticket said. Static sensors tell the agent its code is well-formed; the feature working is a different fact, and only something that opens the product can observe it. What that something has to be is below.

The bug the harness shipped after we'd written it down twice

This summer one of our own agents shipped a change to our pipeline. The spec said one result type must never be offloaded. That constraint was written in the RFC, in the spec, and as a comment inside the code it was changing. The agent read all three and shipped the violation anyway, which is the case for spec-driven verification: the spec that drove the build is the thing to check the result against.

Every scripted check stayed green. The unit tests passed, the linters passed. The only thing that broke was the interactive path, and nothing in the suite exercised it. What caught it was a QA agent using the feature in a preview environment and reporting that it didn't do what the PR said.

When we had the coding agent retrospect its own failure, it wrote: I had the answer written down twice and I shipped the bug anyway. That sentence is the argument for a behaviour sensor. Guides raise the odds of the right behaviour without guaranteeing it, and when you run thousands of agent steps a day, odds are not enough: even a 99% chance of the intended action per step compounds into a near certainty of something wrong somewhere.

Why the static sensors couldn't see it

Agents optimise for reward, not intent. They line up often enough that the difference is easy to miss. When they drift, the agent satisfies the check and misses the point. A linter checks shape. A type checker checks contracts. The agent's own test checks what the agent thought the feature was, and it wrote that test with the same misunderstanding it wrote the code with. A review agent reads the diff and can tell you the code is bad; it cannot tell you the product is broken, because the product isn't in its input.

Birgitta's taxonomy has a second axis I find useful: sensors are computational (deterministic, fast, run by the CPU) or inferential (slower, costlier, non-deterministic). The behaviour sensor is inferential by nature, and I think that's why it gets left out of harness designs. It's the expensive one. But OpenAI's own harness post describes wiring the Chrome DevTools Protocol into the agent runtime so Codex can take DOM snapshots and screenshots and record video of a failure and its fix. That is a behaviour sensor built for the agent's own use, even if the post never calls it one.

What a behaviour sensor has to be

Three questions I ask about any check before it goes in the loop.

Can the agent invoke it itself, without a human? If a person has to click "run", it's a stage gate.

Does it fail with output the agent can act on? A red light is one bit of information. The agent learns something is wrong and nothing about what. Give it the assertion that failed, a screenshot of the moment it broke, the network call that returned a 500, and it has a gradient to follow.

Does it run before a human is involved? A check that happens after review has already spent the expensive resource. The point of the harness is to direct human attention to where it matters, which means the loop has to close on its own first.

Scripted end-to-end tests pretend to meet these criteria and don't. They're a snapshot of something that varies, which is why they flake, and a flaky sensor is worse than none because the agent learns to ignore it. The invariants in a harness have to be things that are as true next year as they are today. "The health badge appears on every surface that shows a deal" is an invariant about behaviour. "Click the element with id badge-7" is a recording.

What ours looks like

I'll describe the one we run, since it's the one I know: PR testing on QA.tech, an agentic QA platform. The sensor is a second agent with no stake in the change. When a PR opens and its preview deploys, it reads what the PR claims to do, picks the goal-based tests that cover the flows the claim touches, adds tests only where nothing covers a flow, runs the set in the preview and posts the result on the PR: a verdict, what was tested, and the per-test table, with the screenshots and step trace behind each row one click away in the run. The verdict can be pass, fail, or unable to verify. The last one is for when the tests ran but none of them crossed the change; a sensor that can say it didn't see enough is worth more than one that defaults to green.

What makes it a sensor is that the output goes back to the coding agent. Over MCP, Claude Code, Cursor or Codex can trigger a run on just the three tests touching the file it edited, read the trace and the screenshots, and fix the failure before anyone opens the PR. It can also tunnel to a local dev server and run the check before committing. That closes the loop on behaviour, the part Birgitta's article says still has a lot to do.

It's bounded. It sees the product the way a user does, through the UI, so a side effect in a batch job is outside it. And it's inferential, so a run costs more than a lint pass; I'd rather spend that on the few changes that need it than spend a person's afternoon on all of them.

Measure the judges too

One last thing from the talk I gave at CTO Craft in September, because it applies to every inferential sensor including ours. An unmeasured judge is a trust, not a verification. If you add a review agent, a test-generating agent, or a behaviour sensor to your harness, calibrate it against a sample you've labelled by hand, track its precision, and watch it for drift. Ask yourself a harder version of the question: when you changed your AGENTS.md last week, can you say whether it made your agents better or worse? If the answer is no, the harness has guides and no sensors on the guides.

OpenAI, Stripe and the others publishing write-ups on agent-written code have stopped trusting the agent's confidence in its own output and built verifiers for the code and for the agent's actions. The running product is the next sensor in that row.

FAQ

What is harness engineering?

Designing the system around a coding agent so it behaves reliably: the instructions and context that steer it (guides) and the checks that observe its output and feed back (sensors). The term comes from OpenAI's February 2026 post on building a product with "0 lines of manually-written code" and Birgitta Böckeler's April 2026 article on martinfowler.com, which gave it the guides-and-sensors structure.

What sensors are in a coding-agent harness?

Linters, type checkers, fast unit tests, structural and architecture tests, dependency rules, coverage, AI code review and human review, plus production-side checks like log anomaly detection. A sensor that runs the deployed product before merge and checks user-facing flows is missing from the standard list.

Why isn't the agent's own test suite a behaviour sensor?

Because the same model wrote both, with the same reading of the ticket. A test the agent writes can only check the agent's version of the intent. Keep it for regressions mid-task and let something else judge the feature.

How is this different from a code review agent?

The input. A review agent gets the diff; the behaviour sensor gets the deployed preview and a user's path through it. A reviewer can tell you the code is wrong in ways that are visible from the code. Whether the feature does what the ticket asked for is only visible when the product runs.

Does the behaviour sensor replace the human reviewer?

Not yet. It changes what lands in front of the reviewer: a verdict and the evidence behind it, with the diff available if they want it. On a low-risk change where every sensor is green, the review check is the merge gate and the human reads the evidence.

Where does this fit in the four verification layers?

It's the fourth. How to verify the work of coding agents walks through static checks, the agent's own tests, review agents and product behaviour, and which pull requests need which.

Your team moves fast. Can your testing keep up?

QA.tech agents test your product autonomously, so moving fast never means shipping broken. See how it works in a 30‑minute demo.

Get a demo