Knowledge Graphs as Context

An agent is asked a question that sounds routine: which services call the payment service, and which of those are owned by a team that is on a deprecation deadline this quarter. The answer exists. It is not hiding — every fact in it is written down somewhere, correctly, by someone who knew what they were talking about. The service catalog lists callers. A README states ownership. A deprecation ticket names the deadline and the team. None of that is wrong, and none of that is missing.

What is missing is the sentence that connects them. No document says "the three services that call payment-service and are owned by a team with a Q3 deprecation deadline are X, Y, and Z" — because no single author was ever positioned to write that sentence. It has to be assembled, on the fly, by reading the catalog, then the README, then the ticket, then holding all three in working memory long enough to intersect them correctly. An agent can do this. It can also do it wrong, silently, by missing one caller or misreading one ownership line — and nothing in the prose it read would have told it so.

The multi-hop tax

This is not a content problem. Every fact an agent needed was present in context, individually correct, individually retrievable. It is a shape problem: prose is organized for narrative reading, one document at a time, and a multi-hop question — find everything two steps removed from X — is not a narrative question. It is also not the context rot Chapter 2 describes: rot is a model’s attention degrading as input grows long, inside a single document. The multi-hop tax shows up even in a short context window with room to spare, because the missing sentence was never written by anyone, at any length. Anthropic’s framing of context engineering treats structure as a first-order lever precisely because the same facts, organized differently, cost a model a different number of reasoning steps to use correctly.

Microsoft Research’s GraphRAG work made the same point from the retrieval side: conventional retrieval, built to fetch the passage most similar to a query, fails specifically on questions that require reasoning across an entire corpus rather than pulling one relevant chunk — the kind of question this chapter opened with. Converting the corpus into entities and relations first, then answering by traversal, was built as a direct response to that failure mode, not as a general upgrade to retrieval.

A 2025 systematic comparison of the two approaches found the split holds in both directions and neither wins outright: retrieval over prose remains stronger on single-hop, detail-oriented, and time-sensitive questions, where the answer is one passage and freshness matters more than structure; graph-based retrieval pulls ahead specifically on multi-hop and relationship-heavy questions, where the value is in the join, not in any one fact. Combining the two at the response stage beat either baseline alone. The honest reading is not "graphs are better" — it is that the two shapes are good at different questions, and most real corpora contain both kinds.

Entities and relations

The basic shape is not exotic. A knowledge graph represents information as entities — a service, a team, a deadline, a decision — connected by typed relations: calls, owns, supersedes, blocks. The same idea works whether it is implemented as a formal graph database with a query language, or as something far lighter: a table of (subject, relation, object) rows, or a set of files with explicit cross-references between them. The formality is a deployment choice. The shape — nameable things, connected by nameable relationships, queryable by traversal rather than search — is the actual idea, and it predates any particular database technology by decades; Designing Data-Intensive Applications treats graph-like data models as one of a small number of fundamentally different ways to organize highly interconnected data, alongside relational and document models, each suited to a different access pattern rather than one superseding the others.

This volume has already built one example without naming it as such. Decision Memory — the practice, described in the previous chapter, of keeping "what was decided, why, and what it superseded" as first-class, queryable records rather than prose buried in a trace — is graph-shaped even when it is stored as flat rows in a table. A decision links to the files it touched and the decision it superseded; asking "why does this file look like this" is a two-hop traversal — decision to file, decision to prior decision — dressed up as a database query. The graph was there before anyone called it one, because the underlying questions — what led to this, what did this replace — were multi-hop questions from the start.

What each shape is good at

The tradeoff is symmetric enough to be worth laying out plainly, by property rather than by verdict:

Neither column is the fallback for when the other is unavailable. A team that graphs everything pays the maintenance cost even on questions prose was already answering fine; a team that never structures anything pays the multi-hop tax on every question that actually needed a join. The decision is per-question-type, not per-project.

The honest tradeoff

Decision Memory’s own design makes the tradeoff concrete rather than abstract. Its "what happened" and "what it superseded" are structured — entity, relation, timestamp — because those are exactly the multi-hop facts a later session needs to traverse quickly. Its "why" stays free text, on purpose. The reasoning behind a decision usually resists clean categorization: it is a paragraph about tradeoffs considered and rejected, constraints that do not fit a fixed vocabulary of relation types, judgment calls that would need a new edge type invented for every decision if forced into structure. Flattening "why" into a relation would not make it more queryable — it would make it wrong, in the specific way that structured data is wrong when it is asked to hold something that was never categorical to begin with.

The "why" behind a decision is a paragraph, not a relation type. Flattening it into structure does not compress it — it deletes the part a human actually needed.

You probably do not need a graph database

The unglamorous conclusion is the useful one. Most teams that would benefit from graph-shaped context do not need to stand up new infrastructure to get it. A lightweight structured index — a handful of files with explicit cross-references, or a simple entity/relation table sitting alongside the plain-file memory the previous chapter described — captures most of the multi-hop-query value at a fraction of the operational cost of a dedicated graph database with its own query language, its own extraction pipeline, and its own on-call rotation. The value was never in the database technology. It was in making the join answerable in one lookup instead of several.

The maintenance cost is not hypothetical, either. Research on knowledge graphs built by automated extraction — the realistic path for most engineering teams, who are not going to hand-author entity/relation pairs — finds that extraction error does not fail loudly. Below some quality threshold, downstream analyses stay close to correct; past it, errors compound in a consistent direction rather than canceling out, and the graph starts confidently answering wrong. A graph nobody is maintaining is not neutral — it decays into a structure that looks authoritative and is quietly lying, which is a worse failure than prose that was merely scattered.

Where this shape already exists

Teams do not need to import graph theory to check whether this argument applies to them. The test is behavioral: how often does a real question — from an agent or a teammate — require joining facts across more than one document to answer, and how often is that join done wrong or skipped. The following is illustrative, not measured — a sketch of the kind of before/after telemetry that should exist before deciding to formalize, not a claim about any specific team’s numbers.

When to formalize

1. Count the joins, not the documents. Track how often a real question requires assembling facts from more than one source before deciding structure is worth the cost.
2. Start with cross-references, not a database. A README that links to the service it depends on, and a ticket that links to the decision it implements, gets most of the traversal benefit for the cost of a habit.
3. Keep "why" as prose even after "what" becomes structured. Decision Memory’s split — structured edges, free-text rationale — is the right default, not a compromise.
4. Budget for upkeep before building the graph, not after. An entity/relation store with no owner degrades into the least trustworthy kind of documentation: structured, confident, and wrong.
5. Reassess per question-type, not once. A codebase that never had multi-hop questions can grow into one as services multiply — the decision to formalize is a standing question, not a one-time architecture review.

None of this is a new rung on the memory ladder this volume built earlier — it sits inside the rung a team already reached. Once plain files start hitting real query limits, the fix is not a bigger context window or a smarter retrieval prompt; it is a structural choice about how the memory a session already writes gets stored, so the next session can traverse it instead of re-reading it end to end.