'You don't need long context': what the compaction paper means for coding agents

An NVIDIA paper finds agents lose little accuracy when they compact context often. For developers tuning compaction in Claude Code, Codex or their own harness.

Picture one engineer at a two-person dev shop running a coding agent through a long dependency upgrade. Two hours in, the session has read dozens of files, run the test suite a few times and ruled out three fixes. Then the context window fills, the tool auto-compacts, and the agent retries a fix it already rejected, because the summary kept the story and dropped the dead end. The usual responses are a bigger window, or turning compaction off and watching the session closely.

A paper from NVIDIA researchers, You don't need long context (arXiv, Nov 3, 2026), argues for the opposite. Compact more often, and write the compaction as a structured handoff rather than a summary. In the authors' harness, Claude Opus 4.5 and GPT 5.6 Sol kept most of their accuracy on SWE-bench Verified and Terminal-Bench while working in contexts of tens of thousands of tokens rather than hundreds of thousands. This is a research result, not a product. But the compaction prompts are printed in the paper, and you can borrow their shape for tools you already use.

What it is

The paper starts from cost. On H100s at $3.85 per GPU-hour (Nebius, Sept 2026), the authors measured that serving at a 256K context window instead of 16K costs 3.6× more per generated token for GLM 5.2, 5.6× for Kimi K2.6 and 7× for Qwen 3 Coder Next. On an NVIDIA DGX Spark, inter-token latency roughly doubled across the measured context lengths for several local coding models. The authors note that retail API prices hide this, because they are quoted per 1M tokens regardless of context length (arXiv). The paper is also listed on Papers with Code and on NVIDIA's Hugging Face page.

The paper compares three ways of managing an agent's history, each run on top of the released mini-swe-agent harness (v2.4). That harness is a minimal loop that issues one Bash action at a time:

Here is how each finding maps to the pain in the lede, per the paper:

Alternatives: our research found no separate pages on competing methods, and the paper does not test the built-in compaction in Claude Code or Codex. The practitioner sources do describe two approaches:

Who it's for

Pick this up if you run long agent sessions and compaction has burned you. Pick it up too if you maintain your own agent harness, since the paper's prompts can be reused as printed, and the regimes are described in enough detail to reimplement in a ReAct-style loop. And pick it up if you run open-weight models locally, where the paper's DGX Spark measurements show memory and latency growing with context.

Skip it if you want a drop-in product: nothing here ships as a tool. Skip it if your sessions rarely hit compaction. And skip it if you rely on smaller open-weight coding models at tight budgets. Per the paper, Qwen 3 Coder Next fell from 53.0% to 3.6% on SWE-bench Verified at 16K/12K, "mostly because its verbose trajectories leave too little room for useful progress between compactions".

Before and after

This is our reading of how a workflow changes, built from the sources.

Before:

  1. Start a long agent session and let history pile up.
  2. Near the limit, the tool writes a summary. Claude Code does this at about 967K tokens on 1M-context models, per SpecWeave.
  3. Anything said only in chat is "at the mercy of the summary" (DEV Community). The specific variable names and exact error messages get compressed away (Claude Fast).
  4. The agent re-discovers or re-tries things it already learned.

After:

  1. Decide in advance what a handoff must carry: objective, established facts, applied changes, verification, dead ends, remaining plan.
  2. Compact earlier and on purpose, steering the content rather than leaving it to the default summary.
  3. Keep large inputs on disk and have the agent search them instead of loading them whole.
  4. Accept that each handoff is lossy, and keep anything critical in files the agent can re-read.

Try it

There is no package to install. The smallest real step is to borrow the paper's handoff structure. This is the mini-swe-agent compaction prompt from Appendix G.1 of the paper, copied as printed. The braces are template variables the authors fill per benchmark.

Stop work. Write only a concise handoff recap; do not run commands or call tools.
The conversation will reset to the system prompt, task, and this recap.
{persistence}
Record:
1. Objective - one line describing the required result.
2. Established facts - exact paths, relevant symbols or commands, observed values,
and important errors/output. Separate observations from unverified ideas.
3. Applied changes - each changed/created file, what changed, and whether verified.
4. Verification - exact checks run, outcomes, failures, obstacles, and workarounds.
5. Dead ends - failed approaches that should not be repeated.
6. Remaining plan - outstanding subgoals, in execution order; the immediate next
command or edit; and the condition for completion.
{handoff_details}
If needed output was truncated, give a narrow command to retrieve it.
{submission_reminder}

In Claude Code, the DEV Community post says you can add a "Compact Instructions" section to your project-root CLAUDE.md, and steer a manual pass with /compact focus on the auth refactor. Using the six-part list above as that section is our suggestion. The paper did not test it.

To compact earlier, SpecWeave reports that Lydia Hallie of the Claude Code team recommends /autocompact 400k. SpecWeave says the valid range is 100K to 1M, and that on 200K models a 400K setting changes nothing. For Codex, the same page points to model_auto_compact_token_limit in ~/.codex/config.toml.

Cost is just tokens. Note the paper's finding that output tokens "are generally elevated when compacting".

Where it breaks

Sources