fix: make automatic compaction durable and recoverablefix: make automatic compaction durable and recoverable

[jiah0231]

Fixes #373

Problem

Compressed requests can become unrecoverable when the next provider call fails
immediately after compaction.

Two related runtime problems also show up in long Agent sessions:

• ongoing Work/Resume can pass the context threshold without compacting until
  the user pauses or sends another message;
• Summarizing chat context can wait indefinitely when the provider stalls.

Root cause

• Auto-compaction updated local state before the compacted checkpoint was
  guaranteed to be published back to Cursor.
• Compacted checkpoints could keep the pre-compaction token count.
• Auto-compaction was effectively one-shot / Start-only instead of following
  revision progress.
• A Resume could reuse stale cross-run provider usage from the pre-compaction
  history and immediately recompact.
• Automatic summary generation had no end-to-end deadline.
• A fixed 10k reserve did not account for configured maximum output.
• Provider usage anchors count projected messages, so canonical message counts
  cannot safely be used for suffix accounting.

Fix

• Publish the compacted checkpoint and wait for its commit barrier before the
  next provider call.

• Re-estimate token usage from the compacted state.

• Rearm automatic compaction after the revision advances instead of limiting a
  run to one compaction.

• Allow automatic compaction at Resume/model boundaries, so long-running Agent
  work does not require a pause or another user message to trigger it.

• On initial Resume, directly estimate the recovered state until the current
  run has fresh provider usage, avoiding stale pre-compaction usage anchors.

• Preserve current upstream projected-message usage-anchor accounting.

• Reserve:

  max(10_000, max_output_tokens + 4096)

  and use an inclusive threshold.

• Reduce automatic summary max output from 4096 to 2048 tokens.

• Add a 30-second total automatic-compaction deadline.

• If the summary provider stalls or fails, cancel the summary cycle, build a
  durable local fallback summary, publish the compacted checkpoint, and
  continue the Agent.

• Share token-estimation logic between automatic thresholding and compacted
  checkpoint accounting.

• Preserve the latest request context across compaction.

Recovery guarantees covered by tests

The regression suite covers the failure sequence from #373:

automatic compaction
-> compacted checkpoint published
-> immediately following provider request fails
-> run stops
-> Resume uses compacted checkpoint
-> unfinished task continues

It verifies that:

• the durable summary is present;
• the current unfinished task is retained;
• compressed-away full history is not restored;
• token usage is reset after compaction.

It also covers an oversized Resume state that automatically compacts without a
new user message.

Stalled compaction test

A dedicated test uses a provider stream that remains permanently pending.

The real timeout path is exercised:

summary provider never returns
-> ~30 second timeout
-> fallback summary
-> compacted checkpoint published
-> Agent continues

This test completes successfully after the actual timeout rather than using a
mocked instant timeout.

Validation

Passed on the branch rebased onto current upstream main:

cargo fmt --all -- --check
cargo test -p cursor-server --test compaction_recovery -- --nocapture
cargo test -p cursor-server --test compaction_timeout -- --nocapture
cargo test -p cursor-server --test compaction -- --nocapture
cargo test -p cursor-server run::engine::tests -- --nocapture
git diff --check

The full cursor-server --all-targets suite also passes with only the existing
AwaitShell assertion skipped.