Organizational Systems

Eighteen months ago, a team under deadline pressure quietly agreed that AI-assisted changes tagged low risk could skip the second reviewer. Nobody wrote it down as a policy; it just became how Tuesdays worked. Two of the four people in that conversation have since left, one moved to a different team, and the manager who suggested it got promoted twice. When a production incident finally traces back to exactly that gap, the postmortem names a clean root cause: an AI-generated change with a subtle validation bug shipped with one reviewer instead of two. What the postmortem does not name, because nobody in the room makes the connection, is that the incident’s actual origin is a policy decision made a year and a half earlier, by people who are no longer around to recognize their own choice.

This is not a story about one team’s bad luck. It is the ordinary way organizations behave: the decisions that matter most — who to hire, which architecture to commit to, which review step to relax under deadline pressure — rarely resolve immediately. They resolve later, sometimes much later, after enough time has passed that the decision and its consequence no longer look like they belong to the same story. The organization that made the call and the organization living with its result are, in every way that matters, not quite the same organization.

The year-long feedback loop

Systems thinking has a name for what just happened, and it is one of the best-supported findings in the discipline: delay is a primary source of counterintuitive system behavior. Donella Meadows, in Thinking in Systems, treats delayed feedback as one of the central reasons complex systems produce results nobody can easily trace back to a cause — not because the causal chain is mysterious, but because by the time the effect arrives, everything else that happened in between has buried it. A supply shortage, a market crash, an engineering incident: the mechanism repeats. The longer the gap between an action and its consequence, the harder it becomes for the people experiencing the consequence to recognize the action that produced it, because they are rarely still the same people, working from the same context, who made the original call.

Chris Argyris opened a 1977 Harvard Business Review article with a version of this same failure that has nothing to do with software: at least five people inside a company knew a struggling product line was in serious trouble roughly six years before the company finally discontinued it. Six years is an unusually long delay, but the shape of the failure is exactly the organizational-systems shape — a signal existed, several people held pieces of it, and nothing in the organization was built to aggregate those pieces into a decision until the cost had compounded for most of a decade. An engineering organization’s version of this rarely takes six years. It takes closer to eighteen months — long enough for the thread to go cold, short enough that most of the people involved are still reachable, if anyone thought to ask them.

Why delay defeats intuition

None of this is a failure of individual judgment. Human intuition is built for fast, local feedback — touch something hot, feel pain immediately, learn not to touch it again — and organizational feedback loops routinely violate every one of those conditions. The consequence is delayed, so its connection to the original cause is severed by time. The consequence is diffuse, showing up as a slow rise in incident rate or a quiet decline in review quality rather than one traceable event. And the consequence often lands on a different part of the organization than the one that made the original decision, so the group feeling the pain rarely has the context to recognize its source. A system whose feedback loops are structurally this slow and this indirect will keep producing decisions that look reasonable in the moment and inexplicable a year later, no matter how careful any individual inside it is being.

The learning organization

Peter Senge gave this problem its most influential name in 1990: the learning organization, one that continually expands its capacity to create the results it actually wants, rather than repeatedly producing the results its current structure is built to generate. The Fifth Discipline rests on five practices — personal mastery, examining mental models, building shared vision, team learning, and systems thinking — with systems thinking positioned as the discipline that integrates the other four, because it is the one that lets people see the structure behind a recurring problem instead of just the latest instance of it. An organization that lacks this capacity does not fail to notice problems. It notices the same problem, freshly, every time it appears, and responds to each occurrence as though it were the first.

Argyris supplied a sharper, more diagnostic mechanism a few years earlier: single-loop versus double-loop learning. Single-loop learning corrects an error without questioning the assumption that produced it — the postmortem’s action item is add a second reviewer, applied once, to this incident. Double-loop learning asks why that assumption was in place at all, and changes the governing rule rather than just the latest instance of it. Both loops close a feedback loop. Only one of them stops the same failure from returning next year wearing a different commit hash and a different postmortem number.

Two organizations, same incident

The practical difference between an organization that learns and one that repeats shows up less in any single decision and more in a handful of structural habits — whether anything exists that would notice a pattern, as opposed to relying on someone happening to remember.

Does anyone notice the pattern?

Apply this directly to AI-assisted engineering and the question sharpens. This Library’s Volume VII chapter on Provenance & Attribution makes a version of the same point at the level of a single postmortem: one incident naming a model failure mode is a data point, and three incidents naming the same failure mode, across different sessions, is a harness bug wearing a disguise. That distinction only does work if something in the organization is positioned to notice the third instance and connect it to the first two. Most engineering organizations today have nothing playing that role for AI-related failures specifically. Each session lives inside its own conversation, each team runs its own postmortem, and the same category of mistake — an agent that reliably drops a particular edge case, a prompt pattern that reliably produces the same kind of hallucinated dependency — can recur across a dozen unrelated projects without anyone ever putting the dozen occurrences in the same room.

The raw material for closing this loop already exists at the session level more often than most teams realize. Structured session records — a progress log, a feature checklist, a git history an agent can use to recover state — are exactly the mechanism Anthropic’s engineering team describes as how a long-running agent session avoids re-discovering the same context from scratch. That is single-session learning, and it works. Far rarer is the equivalent structure one level up: an organization that treats the accumulated record of many sessions, across many teams, as a corpus worth mining for a repeating pattern, the way Google’s site reliability team runs aggregate postmortem tooling and a postmortem-of-the-month review specifically to surface themes no single incident report would show on its own. A team can have excellent memory inside a session and total amnesia across a hundred of them, and from the outside the two look identical — until the pattern finally costs enough to get noticed.

Whether that noticing happens at all depends on the culture underneath it more than the tooling. DORA’s research, building on sociologist Ron Westrum’s typology, ties software delivery performance to whether an organization’s information flow is generative — actively sought, shared, and acted on — or pathological, where inconvenient signals get suppressed rather than surfaced. A pathological or merely bureaucratic culture will not develop a pattern-detection habit for AI failures no matter how good the underlying tooling is, because the habit requires someone to volunteer that something went wrong in a way that might implicate a process they built or defended. DORA’s 2025 report makes the amplifier case explicit for AI specifically.

AI’s primary role is as an amplifier, magnifying an organization’s existing strengths and weaknesses.

DORA, State of AI-assisted Software Development 2025

An organization with a working learning loop gets a faster, cheaper version of that loop once AI-assisted work enters the picture — the same postmortem-aggregation habit just has a new category of finding to look for. An organization without one does not get spared; it gets the same blind spot at higher velocity, because AI-assisted teams generate more sessions, more diffs, and more chances for an identical failure to recur quietly than a purely human team ever did.

What a pattern-detection loop would need to see

None of this requires exotic tooling. It requires deciding, structurally, that the unit worth reviewing is not the incident but the incident type, and building whatever is small enough to actually maintain to make that comparison possible.

What to build on Monday

1. Treat every AI-related postmortem finding as two things: an incident to close and a data point to file against a running list of failure categories — even if that list starts as a spreadsheet.
2. Assign someone, even part time, the job Google’s SRE org gives its aggregate-postmortem tooling: periodically re-reading the last quarter’s incidents specifically looking for a repeat, not a root cause.
3. When a repeat surfaces, resist the single-loop fix. Ask what governing assumption — about review, about which changes count as low risk, about which AI failure modes get caught before merge — allowed both instances, and change that instead of writing one more comment on one more pull request.
4. Make organizational memory outlast team turnover. A record that exists only in the head of someone who has since left was never organizational learning — it was personal memory with an audience.
5. Measure the gap, not just the count. How long between when a pattern started and when someone noticed is itself the signal — a shrinking gap over successive quarters is a learning organization; a flat or growing one is not.

None of this is a claim that an organization needs a formal knowledge-management platform before it can call itself a learning organization. It needs, at minimum, one habit repeated reliably: when something goes wrong, ask not just what happened, but whether this has happened before, under a different name, to a different team, far enough in the past that nobody in the room remembers making the choice that set it up. The next chapter in this volume turns the same systems lens directly onto AI-assisted engineering itself — what it looks like when the amplifier framing borrowed here from DORA is followed all the way through.