Context Health

Three hours into a long session, an engineer notices something is off. The agent proposes changing a function under a name it retired twenty minutes earlier, as if the rename had never happened. Or it re-reads a file it already fixed, treats the on-disk version as ground truth, and quietly reverts its own work. Nobody sounds an alarm, because there is no alarm to sound. The session just starts to feel wrong, the way a colleague’s judgment starts to feel wrong near the end of a long shift — degraded, but not yet failed, and hard to point to precisely. The engineer keeps going anyway, because stopping to diagnose a feeling costs time that fixing an obvious bug does not.

By the time the degradation is obvious enough to act on, it has usually cost real time. The engineer restarts, re-explains the goal, and re-establishes whatever decisions mattered, absorbing the loss as a fact of life rather than a measured event. That is the diagnostic gap this chapter is about: engineers can tell, eventually, that a session has lost the thread, but they can rarely tell early, and they can almost never say precisely why. A vague sense of drift is not a monitoring system, and treating it as one means every warning arrives after the cost is already sunk.

A judgment call, made too late

Chapter 2 established the mechanism: models do not use their context windows uniformly, and reliability degrades unevenly as input grows, well before the window is nominally full. The finding has two independent lines of evidence behind it. An early, widely replicated study found that models retrieve information best from the start and end of a long context and worst from the middle — position, not just volume, shapes what a model actually uses. Chroma Research’s later, more exhaustive testing extended the picture: performance varies significantly with input length even on simple tasks, retrieval accuracy drops as the semantic distance between a question and its answer widens, and individual distractors have outsized, unpredictable effects. None of this shows up as an error. It shows up as an answer that is subtly, confidently wrong.

What teams are left with, absent a way to measure any of this directly, is a feeling. Ask an engineer whether a given session is still trustworthy and the answer arrives instinctively — it’s been going a while, it’s repeating itself, something’s off — the same retrospective judgment call this volume keeps finding underneath AI-assisted work. Volume I named the cost of a session having to reconstruct what it once knew context debt; a health score is what lets a team notice the debt accruing instead of discovering it only after the balance has come due.

Health checks, not vibes

The fix looks like a move engineering organizations already made once, for a different resource. A service’s reliability used to be judged the same retrospective way — is anyone complaining yet — until site reliability engineering replaced that with health computed continuously from several weighted signals rather than any single one deciding alone: latency, traffic, errors, saturation, each contributing to one status. Context deserves the same treatment. A session’s context health is not a boolean, full or not full, but a score assembled from independent signals, several of which can degrade a session entirely on their own.

The distinction that makes this worth building is diagnostic, not descriptive. A bare percentage — this session is at seventy-eight percent of its window — tells an engineer almost nothing actionable. A score of sixty-two, in the warning band, with the reason attached — three files this session read have since changed on disk — tells the engineer exactly what to do next. The number without the reason is a temperature reading. The number with the reason is a runbook, which is the entire argument for building this as a first-class, continuously computed signal instead of a retrospective judgment call.

A red status that names its cause is a runbook. A red status that doesn’t is a rumor.

Four signals, one score

Four factors, each cheap to compute from data a harness already generates, are enough to produce a useful score. None require a new subsystem — they require paying attention to signals a session’s tool calls and file reads already leave behind.

• Utilization — how full the context window is, measured against its effective capacity rather than the number advertised on a model’s spec sheet. Risk ramps as utilization climbs from roughly half full toward nearly full, echoing Chapter 2’s degradation curve. Remediation: compact proactively before the ramp gets steep, rather than waiting for a failure to force the issue.
• Rot signals — files or facts the session read early on that have since changed, on disk or in reality, without the session’s knowledge. A session still reasoning at minute ninety from a config file it read at minute five is working from a stale reference it has no way of knowing is stale. Remediation: re-read the specific files flagged as changed, not the whole codebase.
• Thrash — a session repeating an edit-search-error loop without forward progress, independent of how full its window is. A session at thirty percent utilization can thrash as badly as one at ninety; thrash is a behavioral tell, not a capacity problem. Remediation: stop and restart with a handoff summary rather than pushing further into a loop that is not converging.
• Compaction debt — a session that has already been summarized one or more times. Each compaction is a controlled, deliberate loss, developed in full later in this volume’s chapter on compaction contracts, but it is still a loss. Remediation: treat repeated compaction as a ceiling on the score rather than a one-time deduction, until the session re-grounds itself with fresh reads.

Utilization deserves the emphasis it gets because the naive version of this factor — tokens used divided by tokens available — is the wrong denominator. Anthropic’s own engineering guidance is explicit that context should be treated “as a finite resource with diminishing marginal returns”: models remain capable at longer contexts but show reduced precision on retrieval and reasoning well before they run out of room. Utilization belongs on a session’s effective capacity, not the number printed on a model’s spec sheet, or the score will stay green long after the session has quietly started to struggle.

The rot-signal factor names a problem with its own benchmark literature. LongMemEval, an academic evaluation of long-term conversational memory, tests chat assistants specifically on knowledge updates — information that was true when a system first encountered it and has since changed — and finds accuracy on that category dropping sharply under sustained interaction, even in systems built explicitly for long-term memory. A session reasoning from a file it read an hour ago is running an informal version of the same test, continuously, whether anyone is watching or not.

Compaction debt is the odd factor out, and deliberately so. Compaction is a real mitigation, not a false one — Anthropic’s guidance for long-running agent harnesses treats it as a designed response to a hard limit, not a bug to route around. But a compacted summary does not reliably carry every instruction the original session held, only an approximation of what mattered enough to survive. A health score should treat that approximation as debt the session is still carrying, not a problem it already solved, which is why compaction caps a score rather than merely subtracting from it. A session on its third compaction has been through three rounds of that approximation, each one compounding on the last, and a score that lets three compactions read as healthy as zero is measuring the wrong thing.

From score to band

Rolled together, the four signals produce a single number, and the number resolves to one of three bands. The bands are not decoration — each implies a different action, which is the entire point of computing this continuously instead of waiting for the retrospective judgment call.

The value of the band is that it is never delivered alone. A session in the warning band whose reason is utilization past the effective midpoint, and nothing else degraded, calls for a straightforward proactive compaction. A session in the same band whose reason is two files read at minute ten have since been edited by a separate process calls for two targeted re-reads instead. Same band, sometimes the same number, entirely different fix — which is exactly why the practical implementation guidance in Google’s Site Reliability Workbook insists that alerts carry the symptom that triggered them, not just the fact that a threshold was crossed. A level without a named cause is close to useless as a diagnostic instrument.

What a fleet of sessions should look like

None of the figures below are measured. Operon’s context health monitor computes exactly this kind of score continuously, per session, but this book does not yet have a large enough published sample to report real distributional numbers. What follows is the shape the data should take once it exists — the kind of telemetry any team building or buying a context health check should expect to see, and should be suspicious of a vendor who cannot produce.

What to do when the score drops

1. Compute utilization against effective capacity, not the number on the model’s spec sheet — the degradation curve starts well before the window is full.
2. Track rot signals as a live diff, not a one-time check — a file changed after the session read it is a fact, not a guess, and it should be checked continuously.
3. Give thrash its own signal, independent of utilization — a session repeating a loop at twenty percent capacity is failing for a different reason than one at ninety, and needs a different fix.
4. Treat compaction as debt with a ceiling, not a deduction with a floor — a twice-summarized session should not score as healthy as one still reasoning from its original context, no matter how clean the other three factors look.
5. Attach a reason to every band, always — a score without a named cause tells an engineer to worry; a score with one tells the engineer what to do next.

None of this replaces judgment. An engineer three hours into a session still knows things the score does not. What it replaces is the wait — the health check exists so the warning arrives while a five-minute re-read is still cheaper than a restart, not an hour after the session has already been quietly wrong. It also replaces the argument. A team that disagrees about whether a session is still worth continuing is, underneath the disagreement, usually just missing the same four numbers — and a score with a named reason turns that argument into a five-minute check instead of a standoff. The chapter that follows this one goes further into the loss compaction represents; this chapter has only flagged it as a debt the score should carry until it is paid down.