A Short History of Engineering Metrics

A VP of engineering sits down for a quarterly review and asks a question that sounds simple: is the team more productive than it was a year ago? Three dashboards answer at once, and they do not agree. The delivery dashboard says yes — deploy frequency is up, lead time is down. The engagement survey says no — satisfaction scores have slid for two straight quarters. The engineer sitting closest to the work says neither, because more of her day now goes to reviewing pull requests an agent wrote than to writing code herself, and no dashboard in the room asked about that. All three answers are honest. None of them is the whole picture, and the VP has no way to know which one to trust, because nobody agreed on what "productive" meant before the meeting started.

This is not a new argument, and it did not start with AI. Software engineering has been trying to measure itself since before most of the people in that room were born, and every serious attempt has run into the same wall: the thing that is easy to count is rarely the thing that matters, and the thing that matters is rarely easy to count.

What counting output actually measured

The discipline’s first default answer, through the 1970s and into the 1980s, was lines of code produced per unit time — a number every compiler could report without anyone instrumenting anything. Fred Brooks watched estimation and staffing decisions built on exactly that number fail on the same IBM project he diagnosed in "The Mythical Man-Month," and the book’s most famous claim — that adding programmers to a late project makes it later, because effort and progress are not interchangeable units — is itself an argument against treating output as a proxy for progress. Lines of code counts activity, not the thing the activity was supposed to produce, and it rewards precisely the behavior a careful engineer tries to avoid: a verbose, duplicated, unrefactored solution outscores a terse, well-factored one on the only axis the metric can see.

The metric survived years past the point its credibility held, for a reason worth naming because it recurs throughout this history: it was cheap to compute, and the honest alternative — judging whether the software actually did what it was supposed to, well, which requires someone to look at the work rather than tabulate around it — was not. Every framework in the rest of this chapter is, in one sense, a renewed attempt to find something both informative and cheap enough that an organization will actually keep measuring it.

DORA and the four keys

The next serious attempt arrived four decades later, and it came from research rather than convenience. The DevOps Research and Assessment program, led by Nicole Forsgren, Jez Humble, and Gene Kim, ran multi-year surveys of engineering organizations before condensing what they found into a small set of measures, published in 2018 in "Accelerate" and in DORA’s annual State of DevOps reports: deployment frequency, lead time for changes, change failure rate, and time to restore service. Two measure speed — how often a team ships, and how long a commit takes to reach production. Two measure stability — how often a change breaks something, and how fast the team recovers when it does. The pairing was the finding, not an afterthought: the research showed elite teams were fast and stable at the same time, directly contradicting the old assumption that an organization has to trade one for the other.

The four keys’ real strength is scarcity, chosen deliberately so the numbers check each other. A team that inflates deployment frequency by shipping trivial changes will usually see lead time or change failure rate move in the wrong direction to compensate, because four correlated measures are harder to game in combination than one measure is to game alone. That discipline is why the four keys became the closest thing the industry has to a shared vocabulary for delivery performance — cited in board decks, adopted by tooling vendors who agreed on almost nothing else about how to build software.

The critique, when it came, was not that the four keys were wrong but that they were narrow — honestly scoped that way from the start, and honestly outgrown since. They describe the delivery pipeline: how fast and how safely code moves from commit to production. They say nothing about whether the code was any good, whether the team building it was burning out, or whether the fastest-shipping team was building the right thing at all. A team can hold an excellent DORA scorecard while code review quietly turns into rubber-stamping, technical debt accumulates unmeasured, and satisfaction declines — none of which a deployment-frequency chart will register. The same numbers reward superficial compliance in the other direction, too: a flood of small, low-risk deploys inflates deployment frequency without reflecting any real gain in capability, and nothing in the four keys distinguishes a genuinely elite team from one that keeps its failure rate low by keeping its ambition low. DORA’s own subsequent work concedes the gap; the framework has kept evolving since 2018, most recently reorganized in the 2025 State of DevOps report — itself a tacit admission that four numbers, however well chosen, were never going to be the whole answer.

SPACE: a deliberate corrective

The corrective arrived in 2021, when Nicole Forsgren returned with a different set of coauthors — Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler — publishing "The SPACE of Developer Productivity" in ACM Queue. Their argument was explicit, and aimed as much at DORA-only dashboards as at raw activity counts: developer productivity, they wrote, cannot be captured by a single metric or a single dimension, no matter how well the metric is chosen. SPACE names five: Satisfaction and well-being, Performance, Activity, Communication and collaboration, and Efficiency and flow. Only one of the five — Activity — is the kind of system-generated count that dominated the lines-of-code era and still dominates most engineering dashboards today; the rest require asking people how they feel about their work, tracing how information actually moves across a team, and judging outcomes rather than tallying output.

SPACE’s strength is exactly its multiplicity. Putting self-reported satisfaction on the same dashboard as commit counts makes any single number harder to over-trust — a team cannot look productive on SPACE while its developers are quietly burning out, because burnout is one of the five things the framework asks about directly, not an externality left for someone else to notice later. It also names something DORA’s own scope deliberately excludes: whether work inside a team is actually coordinated, or five people are quietly duplicating each other’s effort while every individual delivery number still looks fine.

The cost of that multiplicity is real, and SPACE’s own authors did not pretend otherwise: the paper describes dimensions to think about, not a fixed set of metrics to install, which leaves each organization to invent its own instrumentation for five different kinds of signal — a far heavier lift than pointing a dashboard at a CI pipeline. The self-reported dimensions are the most fragile in practice. Developers grow wary of satisfaction surveys they suspect will be used against them; teams under scrutiny produce contradictory signals across dimensions rather than one clean story; and problems the framework surfaces — an overloaded team, a broken review process — often land with people who have no authority to fix them. Five dimensions measured honestly beat four numbers measured superficially. Five dimensions measured as an annual checkbox, with a satisfaction score nobody ever acts on, are worse than not measuring at all.

DX Core 4: the recent synthesis

The most recent attempt, introduced by Abi Noda and DX in December 2024, treats DORA and SPACE less as competitors than as two answers to two different questions, and tries to fold both into something a team can actually adopt. DX Core 4 names four dimensions, each with a specific measure attached rather than left for the team to invent: speed, measured as engineer-adjusted output such as diffs or pull requests per engineer; effectiveness, measured through a self-reported developer experience index (DXI); quality, measured as change failure rate; and impact, measured as the share of engineering time spent on new capability rather than everything else. Noda’s own framing of the gap is direct — DORA is prescriptive but narrow, focused on system performance; SPACE is comprehensive but, in his words, difficult to actually do, because it hands a team five dimensions and no starting metrics. Core 4 keeps SPACE’s insistence that dimensions can be in tension rather than all pointing the same direction — quality and speed are treated as a genuine trade-off to watch, not two numbers to both maximize — while keeping DORA’s discipline of naming one specific number per dimension instead of an open-ended survey.

Whether that synthesis holds is still an open question rather than a settled one. Core 4 has not accumulated the years of adversarial use DORA and SPACE have. Its speed proxy — output per engineer — inherits some of lines of code’s old vulnerability to gaming through smaller, more numerous changes, and its self-reported effectiveness index carries the same fragility SPACE’s satisfaction dimension does. Its own creators had already published a reconsideration of the framework in light of AI-assisted development within a year of introducing it — itself evidence that nothing in this history stays settled for long.

Three frameworks, read side by side

Laid next to each other, the pattern across five decades is consistent enough to state as a rule: every framework that gains real adoption trades some completeness for adoptability, and every framework that stays honest about its own scope outlives the ones that pretend to be complete.

What the rest of this volume assumes

None of the three frameworks above is presented here as solved, and that is deliberate. Each is real, each is still in active use somewhere right now, and each carries a documented failure mode serious enough that treating it as beyond question would be its own kind of error. Read together they are more useful than any one of them alone. DORA proved a small set of correlated numbers beats one number, and is genuinely hard to fake in combination. SPACE proved that leaving out self-reported experience produces a dashboard that can be technically excellent and organizationally hollow at the same time. DX Core 4 proved that a framework has to be adoptable inside a single planning cycle or most teams will quietly default back to whatever they were already tracking. That is the vocabulary the rest of this volume assumes a reader already has.

The next chapter picks up exactly where DORA’s own most recent work left off: the DORA AI Capabilities Model, published as part of the 2025 State of AI-assisted Software Development report, which this Library has already cited elsewhere as evidence that AI tends to amplify whatever strengths and dysfunctions an organization already had rather than replacing either. The chapters after that turn the same skepticism this history earns onto ground none of these three frameworks was built to answer: what quality means when an agent wrote the diff, whether an organization’s decisions are getting better or only faster, whether anything like SPACE’s Communication and collaboration dimension survives when one side of the collaboration is a model, and how to measure AI itself without falling into the trap lines of code set decades ago — counting what was generated instead of judging whether it was any good.