Context Compaction for Agents: Keeping Long-Horizon Sessions Coherent Past the Window

A coding agent three hours into a database migration proposes switching the ORM adapter. Reasonable idea. It also tried it at hour two, hit a failing integration test, and abandoned the approach. Nobody told it that, because the summarizer that ran forty minutes earlier kept the file tree, the branch name, and a tidy paragraph about "working on the migration," and dropped the one sentence that mattered: this path is closed.

Nothing crashed. The model did not hallucinate. Token usage looked healthy. The agent just burned forty minutes rediscovering its own dead end, and the only way anyone found out was reading the trace afterward.

This is the dominant failure mode in long-horizon agent runs, and it is not a model problem. It is a policy problem, and most teams do not have a policy.

Compaction is a compression policy, not a cleanup job

Three mechanisms get lumped together under "context management," and they fail in different ways.

Trimming drops the oldest messages when the window fills. Cheap, deterministic, and it removes exactly the wrong things, since early turns usually contain the task definition and constraints.

Summarization rewrites history into prose and replaces it. Preserves more, but the summarizer is a second model with its own objective, and by default that objective is "write a good summary" rather than "preserve what the next 200 turns need."

Externalization writes state to a file, a scratchpad, or a store, and keeps a pointer in context. This is what MemGPT formalized in 2023 by treating the context window like RAM and everything else like disk, with the agent responsible for paging. It scales furthest and costs the most engineering.

Real systems use all three. The mistake is running them without a stated objective, so that whether the agent remembers a rejected approach depends on how a summarizer happened to feel about it.

The tempting alternative is to skip compaction entirely and buy a bigger window. That does not work as well as the spec sheets imply. Lost in the Middle showed models retrieve reliably from the start and end of long inputs while accuracy sags in the middle. Chroma's context rot evaluation extended this across 18 models and found performance degrading with input length even on tasks a short prompt handles trivially. Usable context is meaningfully smaller than advertised context, and the gap widens as the task gets harder.

So compaction is not something you do because you ran out of room. You do it because a smaller, better-organized context outperforms a larger, noisier one.

Keep decisions, commitments, and state. Drop the story.

Here is the objective worth writing down before you touch a summarizer prompt. Four buckets, in descending order of value per token.

Decisions with rationale. What was chosen, what was rejected, and why. Rejected options are the highest-value, lowest-cost thing in the entire history, and they are the first casualty of a narrative summarizer, because "we tried X and it did not work" reads like a detour rather than a result. It is a result. It is the most expensive information the agent has produced.

Open commitments. Anything the agent said it would do and has not done. "I will circle back to the auth tests." "Still need to update the changelog." An explicit ledger of unfulfilled promises, updated on every compaction, survives where a summary paragraph will not.

Environment and tool state. Branch name, working directory, ticket IDs, cursor positions, which files are already modified, which credentials are scoped to what. This is small, concrete, and catastrophic to lose. An agent that forgets it already committed a change will happily commit it again.

Raw transcript. Almost entirely disposable once the first three buckets are extracted. Tool output that has already been acted on is the single largest source of dead tokens in most agent runs.

Anthropic's writeup on effective context engineering makes a related argument: treat context as a scarce resource and curate for the smallest set of high-signal tokens. The framing we use with clients is stricter. Compaction is lossy compression with a defined loss function, and the loss function is "would the agent make a different decision on the next turn." If the answer is no, drop it.

The practical version fits in a structured object, not prose:

{
  "goal": "Migrate orders service from ORM v2 to v3",
  "decisions": [
    {"choice": "keep raw SQL for reporting queries", "why": "v3 query builder drops window function support"},
    {"rejected": "swap adapter layer wholesale", "why": "integration test suite fails on connection pooling; 40min spent"}
  ],
  "open_commitments": ["update CHANGELOG", "re-run load test after index change"],
  "env": {"branch": "migrate/orm-v3", "modified": ["db/orders.py", "db/pool.py"], "tests_last_green": "2026-08-13T14:02Z"}
}

Prose summaries lose the rejection entry. Structured state does not, because the schema forces the question.

Compact at boundaries, not at thresholds

Most implementations fire compaction when context hits some percentage of the window. That is a memory-pressure trigger applied to a semantic problem, and it produces the worst possible timing roughly whenever the task is interesting.

Compacting mid-tool-call means the agent loses the reason it made the call. Compacting mid-reasoning-chain means it loses the chain and restarts from a summary of its own half-finished thought. Compacting during a debugging loop means it forgets the last three hypotheses it eliminated - the exact scenario from the opening.

Better triggers are events with natural closure:

• A subtask in the plan reaches a terminal state (done, blocked, or abandoned)
• A test suite or build completes
• A tool call resolves and its output has been acted on
• A human approval gate clears
• The agent is about to enter a phase with different working state

There is a cost argument here too. Rewriting the front of the context invalidates the KV cache prefix, so the turn after compaction pays full prefill. Compact three times during a tight loop and you pay that three times while also destroying continuity. Compact once at a clean boundary and you pay it once at the moment the agent needed a fresh working set anyway.

Cognition's writeup on why they avoid multi-agent architectures lands on an adjacent point from a different direction: context fragmentation, not model capability, is what breaks long agent runs, and every boundary where context gets rewritten or split is a place decisions go missing. The lesson generalizes. Fewer, better-timed compactions beat frequent ones.

Threshold triggers still belong in the system as a backstop. They should be the fallback, not the policy.

Test it with resume-from-checkpoint, not vibes

Compaction quality is measurable, and almost nobody measures it. The test that works is a fork.

1. Instrument real runs to snapshot full state immediately before each compaction event.
2. Fork the run. Branch A continues with complete, uncompacted history. Branch B continues from the compacted state.
3. Run both to completion against the same environment.
4. Compare.

The metrics that matter are not "summary quality." They are behavioral:

Supplement the fork with direct probes injected right after compaction: "List every approach you have already ruled out and why." "What are you currently committed to finishing?" These are cheap, they run in seconds, and they fail loudly. LongMemEval offers a useful taxonomy to borrow here, with roughly 500 curated questions spanning information extraction, multi-session reasoning, temporal reasoning, and knowledge updates. Knowledge updates are the category most relevant to agents and the one summarizers handle worst, because they require remembering that a previously true fact is now false.

Repeated-work rate is the metric we would keep if forced to pick one. It is the direct financial expression of bad compaction, it correlates with everything else on the list, and it is trivially auditable from traces. It also connects to a broader point about why agents fail in production in ways that unit tests never surface.

Frameworks give you the mechanism. The policy is still yours.

Every serious framework now ships compaction primitives, and teams routinely mistake the primitive for a solution.

LangGraph's persistence layer checkpoints graph state at every superstep, which gives durable resume, time travel, and human-in-the-loop interrupts. What it does not give you is a smaller message list. If your state schema holds an append-only list of messages, checkpointing faithfully persists an ever-growing context. Persistence and compaction are orthogonal, and conflating them is the most common architecture mistake we see in code review.

Coding agents with automatic compaction are further along, since they compress at natural points and preserve file state. They still optimize for a generic objective. If your domain has state that a generic summarizer would not recognize as important - a compliance determination, an approval that is scoped to one specific request, a customer commitment made in an earlier turn - a general-purpose summarizer will flatten it into background.

Paging architectures in the MemGPT lineage push the most control to the agent and demand the most engineering, plus they introduce a failure mode of their own: the agent has the information available and does not think to retrieve it. Availability is not recall. That distinction sits at the center of how we think about agent memory beyond RAG, where short-term, long-term, and episodic memory each need their own retention rules.

Pick a framework for its checkpoint and interrupt story. Write the compaction policy yourself, in your domain's vocabulary, and version it like any other production artifact.

What to build this week

A compaction spec is a short document plus a schema. It should name the state buckets you preserve, the schema each one serializes to, the trigger events, the fallback threshold, and the eval suite that guards it. Then wire it in:

• Replace the default summarizer prompt with one that fills your schema and explicitly asks for rejected options and unfulfilled commitments
• Move the trigger from token percentage to boundary events, keeping the threshold as backstop
• Log every compaction with before and after state so failures are reconstructable
• Add three resume probes to CI and fail the build when the repeated-work rate regresses

That is a week of work for most teams, and it converts an invisible reliability tax into a tracked number.

How OpenNash Can Help

We build production agents where long-running sessions are the normal case rather than the demo case, and compaction policy is part of the design phase, not a patch after the first bad rollout. In an audit we trace real runs, quantify repeated work and dropped commitments, and identify where context is being lost. In design we specify the state schema, trigger events, and human approval gates. In build and deployment we ship the policy with a resume-from-checkpoint eval suite, full observability on compaction events, and complete ownership handoff, so your team can change the policy without calling us.

If a platform vendor already covers your use case with acceptable memory behavior, use the platform. If your agents run for hours, carry domain state a generic summarizer cannot recognize, or need auditable decision history for compliance, custom is the honest answer. If you are still in prototype and sessions run under twenty turns, wait. You do not have this problem yet.

Book a call to map this to your workflow.

The teams shipping reliable multi-hour agents in 2026 are not the ones with the largest context windows. They are the ones who decided, deliberately and in writing, what their agents are allowed to forget - and then wrote a test to prove it.