Long-Running & Background Agents

An engineer sizes a task for a weekend, not an afternoon: rip the legacy job queue out of a service and replace it end to end across roughly four hundred call sites, migrate the tests, leave the branch in a state a reviewer can approve Monday morning. They kick off an agent Friday evening, watch it work through the first few files, and go to bed expecting to wake up to a finished branch. What they actually get, if the harness is just a longer version of a normal session, is something quietly worse: an agent that, somewhere around hour four, has forgotten which files it already touched, is second-guessing a decision it made and reverted itself, or is describing file contents it read six hours and several thousand tokens ago as though they were still current.

The instinct is to treat this as a stamina problem — get a bigger context window, tune compaction more aggressively, hope the next model degrades more gracefully. Those changes help at the margins. They do not touch the actual constraint, which the previous chapter already named: an agent session is not an elastic container that stretches to fit the work; it is one window in a sequence of windows, and no amount of stretching a single window turns it into an unbounded one. A harness built for work that spans hours or days has to be designed around context resets happening, not around trying to postpone them indefinitely.

What breaks when a session runs long

The task lengths involved are not hypothetical. METR’s tracking of how long a task frontier models can complete autonomously found the length climbing on a consistent exponential trend, doubling roughly every seven months over several years of models — the kind of curve that turns “a few minutes of work” into “a genuine multi-hour project” inside a couple of model generations. Anthropic’s engineering team, describing the harness pattern behind its own long-running coding-agent experiments, states the resulting design constraint plainly: because context windows are limited, and because most complex projects cannot be completed within a single window, agents need a way to bridge the gap between coding sessions. The sharper version of that problem is that each new session starts with no memory of what came before it — not degraded memory, none. A fresh context window is not a tired engineer who needs a coffee; it is a different engineer who has never seen the codebase, handed the keys and told to continue where someone else left off, with no handoff meeting and no notes unless the harness deliberately wrote some.

The initializer and the incremental agent

Anthropic names its fix a two-role pattern: an initializer that runs once, and a sequence of incremental agents that each pick up where the last one stopped. Readers who met the Ralph loop in this volume’s chapter on the agent loop will recognize the shape underneath it — Geoffrey Huntley’s bash while-loop, feeding an agent’s own output back into a fresh invocation, is the same core idea in its rawest form. In its purest form, as Huntley put it, Ralph is just a loop: `while :; do cat PROMPT.md | claude-code ; done`. What the initializer-plus-incremental split adds is not a new idea so much as real infrastructure wrapped around an old one — the durable state and checkpointing machinery the previous chapter covered, applied deliberately instead of accumulated by accident.

The initializer’s job is entirely setup, and it runs exactly once. In Anthropic’s worked example — an agent asked to clone a complex web application from scratch — the initializer writes a feature list as a structured file with every required feature marked failing, writes a progress log that later agents will append to, makes an initial commit so there is a git history to read, and writes a startup script so the next agent does not have to rediscover how to run the thing it is building. None of this is code the finished project needs; all of it is scaffolding the next agent needs in order to not start from zero.

Every agent after that follows the same short contract, in Anthropic’s framing: read the git log, the progress file, and the feature list to reconstruct where things stand; run the app and do a basic health check before touching anything; make progress on one bounded piece of work rather than many at once; commit with a message that explains what changed and why; update the progress file and the feature list before exiting. One rule inside that loop is explicitly non-negotiable — it is unacceptable for an agent to remove or edit a test to make it stop failing, because that converts an honest signal of incomplete work into a silent lie the next agent, and the human reading the diff Monday morning, will both believe.

What each cycle owes the next

The pattern generalizes past Anthropic’s specific file names. What matters is that a small, fixed set of artifacts carries state across the gap between sessions, and that every incremental agent both reads and updates all of them before it exits — a version of the file-based memory this library’s second volume already covers, made mandatory rather than optional.

That last row is worth dwelling on. An incremental agent that skips the health check is optimizing for its own session looking productive, at the cost of the next session inheriting a broken foundation it has no way to detect until deep into its own work. The exit routine — commit, log, keep the feature list honest — is what makes one session’s output trustworthy enough for the next fresh context to build on without re-verifying everything from scratch. Skip it, and the pattern degrades back into a plain long session with extra file writes, which is exactly the failure mode it was built to avoid.

Inside a long run, illustrated

Operon’s own worktree and checkpoint machinery exists for close to this exact handoff — a session that ends mid-task preserves its worktree rather than discarding it, and a checkpoint gate can pause an incremental agent for review before it commits to the next irreversible step. There is no published telemetry yet on multi-cycle overnight runs specifically, so the figures below are illustrative — the shape of the data a team running this pattern seriously should be collecting, not a measured result.

Iterate deeper, or scale wider

Once a team has a working incremental loop, a genuinely open question follows: is it better to run one agent through many more cycles on the same hard problem, or split the problem and run several agents on the pieces at once? The iterate-deeper case, argued most directly by developer Addy Osmani, is that it is often more fruitful to let one capable agent loop run longer — overnight or across multiple days — on a complex project than to spin up a swarm that is hard to manage. The underlying claim is that many real engineering problems do not decompose cleanly: a migration that touches shared state, or a refactor where every file’s correct shape depends on a decision made in another file, resists being split into independent slices, and an agent that has spent six cycles building a coherent model of the codebase has something a fresh agent starting on a quarter of the problem does not.

It’s often more fruitful to let one capable agent loop run longer — overnight or multiple days — on a complex project than to spin up a swarm that’s hard to manage.

Addy Osmani, on single-agent versus parallel-swarm coding loops

The scale-wider case does not deny any of that; it argues wall-clock time is its own cost, and that for problems that genuinely decompose — a batch of independent bug fixes, a directory of files needing the same mechanical migration applied — four agents run in parallel for two hours can beat one agent run for eight, and an organization with compute budget can trade money for calendar time in a way a solo developer cannot. Both positions turn out to be compatible with the same underlying fact rather than contradicting each other: a controlled Google Research study of agent coordination found multi-agent setups improving performance by roughly 80% on genuinely parallelizable tasks, while degrading it by 39 to 70% on sequential ones, using the same benchmark suite and the same underlying models. The debate is not really iterate-deeper versus scale-wider in the abstract; it is a question of whether a given problem is one thing or several independent things, and that is not always obvious before the work starts. This library returns to parallel and multi-agent architectures directly in a later volume; the point worth holding onto here is narrower — nothing about the pattern in this chapter resolves that debate, and no consensus existed as of mid-2026 on which default a team should reach for.

Before you turn this on overnight

An unattended run that costs nothing to leave going overnight is an appealing story, and it is not quite true. Every incremental cycle is a fresh context window paying its own setup cost — re-reading the progress file, the feature list, enough of the codebase to pass its own health check — before it does a single unit of new work, and a harness that is heavier on planning and file-reading per cycle spends real money doing that re-orientation eight or ten times a night instead of once. Benchmarks running the identical task through different agent harnesses have already found the same work costing several times more in one tool than another, purely as a function of how much re-reading and re-planning the harness does per cycle. The next chapter, on harness economics, works through that tradeoff directly: how much of a long run’s token spend is genuinely new work versus repeated re-grounding, and why two harnesses running the identical overnight loop can differ by several multiples in cost for reasons that have nothing to do with which model either one calls.

None of this argues against the pattern. A well-built initializer-plus-incremental loop is a legitimate way to point real engineering hours at a problem while nobody is watching, and the alternative — a single session quietly running out of context at three in the morning with no record of what it tried — is worse in every way that matters. It argues for treating overnight and long-running as a deliberate harness decision with a cost and a verification burden attached, not a checkbox flipped because a task looked too big to fit in an afternoon.