Engineering Documentation

A platform team’s onboarding doc for its checkout service is, by any human standard, good writing. It opens with two paragraphs of history — why the service exists, what broke before it did — then walks a new hire through the codebase the way a senior engineer would walk them through it in person: conversationally, assuming the reader already knows what an idempotency key is, mentioning the retry convention once, in passing, inside a sentence about a 2024 incident, and trusting the reader to connect it to the retry logic they will see two files later. A new hire reads it in ten minutes and picks up the unstated rule the way people pick up unstated rules — by noticing it is consistent with everything else in the codebase and inferring that it must be deliberate.

An AI agent given the same document as the first file in a new session does not do that. It reads the sentence about the 2024 incident and the sentence about the service’s founding with exactly the same weight, because it has no tacit sense of which line in a friendly narrative doc is the operative instruction and which is scene-setting. Two sessions later, working on a code path near — but not exactly matching — the sentence that carried the rule, it writes a retry loop that ignores the convention, confidently, because nothing in the text told it not to. The doc did not get worse. Its second reader simply arrived without the tacit context the first reader always brought for free.

The reader nobody designed for

This is not a hypothetical mismatch. By early 2026, a sample of roughly 790 million requests across documentation sites hosted on Mintlify, measured over thirty days by user-agent header, put AI agent traffic at 45.3% of total requests — nearly matching human browser traffic at 45.8%, and, per the same analysis, almost certainly an undercount, since not every agent sends an identifiable header. Claude Code alone generated more requests against those docs than Chrome running on Windows did. Documentation was written, for essentially the entire history of the practice, on the assumption that its reader was a person: someone who could skim, infer, tolerate an implied convention, and forgive a paragraph that existed mainly to make the reader feel welcome. That assumption is now wrong for something close to half of the traffic hitting a typical docs site, and most teams’ documentation habits have not caught up to the shift.

What each reader actually needs

The two readers do not just tolerate different styles; they frequently need close to opposite things from the same page. A person benefits from narrative — context that explains why a rule exists tends to make the rule easier to remember and apply correctly in a case the doc never explicitly covered. An agent benefits from structure it can parse without inference: a consistent heading pattern, a convention stated once and never only implied, an example rather than a description of what correct output looks like. Fern, a documentation-tooling vendor whose own product now serves both audiences, put the practical version of this plainly in a March 2026 guide aimed at API teams: an agent needs “clean hierarchies and predictable structures” and direct statements — “OAuth 2.0 authentication” rather than a vague callback to “the authentication method mentioned earlier” — where a human reader tolerates exactly that kind of backward reference without noticing it costs anything.

The hidden pattern, and why the obvious fix is not free

The obvious response is to rewrite documentation the way that kind of guidance recommends — explicit, structured, machine-parseable — and treat the narrative version as a legacy format to retire. The evidence for doing that everywhere is thinner than the confident tone of most vendor guidance suggests. A February 2026 study benchmarking AGENTS.md-style context files across twelve real repositories and four coding agents found that files generated by an LLM — a common, cheap way teams populate them — actually reduced task success by roughly 2% on average while increasing inference cost by more than 20%, largely because the generated files mostly duplicated documentation the agent could already discover by reading the repository directly. Human-written files fared better, but only barely, adding roughly four percentage points of task success at a similar cost premium. A separate empirical survey of real AGENTS.md and CLAUDE.md files in the wild, published a few months earlier, found these files varying enormously in quality and specificity across projects — some genuinely load-bearing, many others closer to boilerplate copied from a template. The honest read of this evidence is not that agent-optimized documentation is worthless. It is that the mere existence of a structured file is not what helps: a file that restates what an agent could already find on its own is closer to this Library’s Context Debt than to genuine standing context — added tokens, added drift risk, no added information.

The failure runs in the other direction too. A doc rewritten entirely as exhaustive, explicit, machine-parseable structure — every parameter typed, every implicit connection spelled out, every section a complete thought retrievable on its own — is precisely the kind of document a human stops reading. It is correct, and it is also the reference manual nobody opens voluntarily, the one engineers route around by asking a colleague instead, which quietly reintroduces the exact problem documentation exists to solve: the real answer lives in someone’s head again, just with an extra, unread page sitting next to it.

Layered, not rewritten

What a growing number of teams are converging on instead is not one document trying to serve both readers at once, but two documents doing two different jobs. This Library’s third volume already named the shape of the first layer: a concise, explicit, standing intent file — a CLAUDE.md, an AGENTS.md, or the org-and-project layered hierarchy the Constitution Pattern and Intent Files chapters describe in depth — loaded unconditionally at the start of every session, written the way the research above suggests actually helps: sparse, load-bearing, stating only what a session could not otherwise discover cheaply. The second layer is the narrative material — the architecture doc, the incident postmortem, the “why we chose this” explainer a new human hire genuinely benefits from reading start to finish — kept available but not force-fed into every session’s starting context. That second layer is retrieved the way this Library’s second volume already described for task-specific context in general: on demand, when the work at hand actually calls for it, rather than loaded up front on the chance it might matter. Anthropic’s own engineering guidance frames the underlying tradeoff without needing to pick one side for every case: standing content earns unconditional loading because it is small and needed everywhere, while everything else waits for a tool call that fetches it exactly when the task requires it.

Sorting a doc into a layer

The failure mode staleness always was

None of this is new in the sense of being undiscovered. Documentation staleness — a doc that once matched the system it described and quietly stopped — is one of the best-studied maladies in software engineering, not a problem AI introduced. A large empirical study mining hundreds of real documentation-related complaints and surveying practitioners, published at ICSE in 2020, built a detailed taxonomy of the ways documentation degrades in practice and found obsolete and outdated content among the issues developers raised most consistently, alongside missing and inconsistent content. Every team that has ever onboarded someone off a six-month-old wiki page already knows the shape of the problem: it does not announce itself, it just sits there being wrong until someone trips on it.

What changes with an AI agent as the reader is not the rot itself; it is what happens when someone trips on it. A human onboarding off a stale doc brings a working skepticism filter that this Library’s Trust Calibration chapter, in a different context, calls calibration — a felt sense, however imperfect, that a doc this old might not be right, worth a quick check against the actual code before acting on it. An agent starting a session has no such felt sense unless something builds it one. It reads a stale architecture doc and a fresh one with identical confidence, because confidence, for a model, is not modulated by “when was this last true” unless that signal is made explicit and checked — the same automation-bias mechanism the Trust Calibration chapter documents in reviewers who stop reading diffs, here acting on the input side instead of the output side.

A stale doc is not just wrong information anymore. It is wrong information delivered with the same unearned authority as everything true sitting right next to it in the file.

The industry’s response to this is early but visible. A short 2026 research note proposed adapting the document-consistency techniques teams already use for ordinary documentation — checking whether a referenced file, function, or config value still exists — specifically to flag drift in AI configuration artifacts like CLAUDE.md and AGENTS.md files, on the premise that a standing file read at the start of every single session is exactly the artifact whose staleness does the most damage per stale sentence, precisely because of how uniformly it gets trusted. Treating an intent file’s freshness as something to actually check, rather than assume, is the practical answer available today: a last-reviewed date a session can see, a lightweight check that a path the file references still exists, an owner who actually gets pinged when the code underneath it changes.

None of this argues for fewer standing files or less documentation generally. It argues for treating a standing intent file the way this volume’s earlier chapters treat any compounding asset: worth writing well, worth keeping current, and worth auditing on a cadence rather than trusting because it exists. The chapters ahead extend the same discipline outward — from one project’s layered docs to a knowledge graph connecting decisions across many, to playbooks that convert one team’s hard-won pattern into another team’s starting point, and eventually to knowledge deliberately written by agents rather than only consumed by them.