Cost Per Outcome

Picture a quarterly engineering review: two peer teams present their AI-assisted development spend to the same VP. Team A’s invoice for the quarter comes to $38,000 — lean, disciplined, the kind of number a finance partner likes to see. Team B’s comes to $91,000, more than double. Left there, the meeting has an obvious next item: find out what Team B is doing wrong and bring it in line with Team A. Nobody in the room asks the question that would actually settle anything — of everything each team shipped, how much of it is still standing.

Suppose someone asks anyway. Team A merged fourteen features that quarter; three were reverted within a month for regressions the review process missed under deadline pressure, and two more were abandoned half-built when the assigned engineer rotated onto another project. Eleven of fourteen survived. Team B merged thirty-one features, one of which was reverted. Thirty of thirty-one survived. Team A’s dollars-per-quarter figure looks better in isolation and worse in every way that matters once a denominator is attached: roughly $3,450 per durable merge against roughly $3,030. The VP, managing the invoice instead of the fraction, was one follow-up question away from rewarding the worse-performing team.

What harness variance reveals

This is not an asymmetry invented to make a point. Benchmarks that hold the underlying model constant and vary only the agent harness around it — how much planning the agent does before acting, how much of the surrounding codebase it reads before proposing a change, how many verification passes it runs before calling the work done — find the identical task costing several times more in one harness than another. A harness tuned to look cheap on the first pass, by skimming context and skipping verification, is frequently the more expensive one once its retries and its failed reviews are counted back against it. The token bill for that first pass simply never saw the rest of the cost.

The mechanism behind the spread is not exotic. Anthropic’s account of building a multi-agent research system found that token usage alone explained roughly 80% of the variance in how well an agentic system performed a task — agentic workflows run about four times the token volume of an equivalent chat exchange, multi-agent workflows about fifteen times, and the additional spend correlated with additional capability rather than waste. Read against the harness-variance finding, the implication sharpens: minimizing tokens with no regard for outcome is not cost discipline. It is a direct trade against the thing that makes an outcome likely in the first place.

The cheap harness and the efficient harness are not the same thing. Only one of them survives contact with a denominator.

The metric everyone reaches for

None of this is visible from inside a token dashboard, which is exactly the trap. Dollars per session, dollars per developer, dollars per month — each of these is legible, real-time, and denominated in a currency finance already understands. None of them contains a merge, a revert, or an abandoned branch. A team can watch its per-session average fall for an entire quarter, report the trend with pride, and never notice that the share of sessions ending in a durable, still-standing change fell even faster.

The pattern engineers run into but rarely name is this: an AI-assisted engineering budget behaves like an investment portfolio, not an expense line. An expense line is judged by minimizing it. A portfolio is judged by its return per dollar deployed, and a portfolio manager who bragged about spending less without reporting the return would be managed out, not promoted. Most AI-cost reporting in 2026 still reads like the expense-line version — precise about the outflow, silent about the return, and confident that silence is the same thing as good news.

Cost per outcome

The fix is a change of unit, not a change of philosophy: stop reporting cost, and start reporting cost per outcome — total AI-assisted development spend divided by a count of outcomes that actually happened. Outcome, here, can mean a merged feature, a merged pull request, or a resolved issue, depending on what a team already tracks well; the framing travels across all three. The numerator is the part every finance team already has, to the cent. The denominator is the part almost nobody has instrumented, and it is where the real work of this metric lives — a sloppy denominator produces a number that looks exactly as rigorous as a good one and measures nothing at all.

Building the ratio

Building this ratio at the organization level comes down to three decisions. Each has a tempting shortcut and a sturdier choice that costs a small amount of additional instrumentation — instrumentation most teams already have the raw data for, sitting unused in version control, in the review queue, and in whatever incident channel logs the reverts. None of the three decisions requires a new tool. All three require someone to actually join the tables that already exist, which is a governance choice more than a technical one, and usually the reason this metric does not already exist at most organizations that could build it this afternoon.

The thirty-day no-revert window matters more than it looks. A pull request that merges and reverts within a week was never really an outcome; it was a false positive that happened to pass CI on the way through. Counting it as a win inflates the denominator with debt that has not come due yet — the same durable-versus-not distinction this volume treats at the level of a single sitting under the name Session ROI. Counting outcomes before they have had time to prove durable is the single most common way this metric gets built wrong on the first attempt.

Where the ratio gets gamed

Cost per merged outcome is a real improvement over cost alone, and it is still gameable, in a specific and predictable direction: split large changes into many small ones, and the denominator inflates without the numerator moving much, because small changes tend to cost less per unit even under generous accounting. A team quietly incentivized on raw merge count discovers this within a sprint or two, usually without anyone deciding to game anything on purpose — trivial refactors and one-line fixes simply start looking more attractive to pick up than the harder feature that would actually move the product, because they move the number that is being watched.

The two partial fixes are the ones already in the table above: weight by change size — lines touched, files changed, or a complexity proxy the team already computes — or weight by reviewer-assessed significance, a lightweight tag a human attaches at merge time (trivial, incremental, substantial). Neither is clean. Size correlates poorly with value; a ten-line fix to a billing bug can matter more than a thousand-line refactor nobody asked for. Reviewer judgment is subjective, and it adds friction to the exact process the metric is trying to describe. This is not a flaw unique to this metric — it is close to why the DORA research program builds a capability model out of several indicators rather than crowning a single number, and why any one ratio, this one included, is a starting instrument rather than a finished one.

What the telemetry should look like

None of this needs to be sophisticated to be useful — a monthly rollup with three columns gets most teams most of the way there, and a spreadsheet updated by hand once a month beats an elaborate dashboard nobody trusts enough to act on. The figures below are not a measured result; they recap the opening scenario in round numbers, sized to be the shape a real dashboard should be checked against, not copied from. The point of publishing an illustrative shape rather than a precise one is to keep the emphasis on the ratio’s structure — spend over durable outcomes, tracked over time — rather than on any specific number this chapter did not measure.

What changes if you use this

What changes in practice is not the invoice itself — it is the sentence the invoice gets to appear in. “We spent $91,000” is a fact with no verb attached. “We spent $91,000 and it produced thirty durable merges, at roughly $3,030 each, down from $3,300 last quarter” is a sentence a VP can act on: keep funding the team getting cheaper per outcome even as its total bill grows, and go find out what the smaller-invoice team is actually doing with the dollars it saved.

This is also, not incidentally, an argument against pure cost minimization as a leadership goal. Anthropic’s variance finding says spend and capability are largely yoked together; the harness-benchmark spread says architecture, not stinginess, is what actually separates an efficient harness from a merely cheap one. Minimizing cost per outcome sometimes means spending more, deliberately, on the layer of the work — planning, context-reading, verification — that the cheap-looking configuration skipped in order to look cheap. The invoice was never the thing to optimize. It was always the numerator.