feat(copilot): read per-request input/cache from session-store.db

CHANGELOG.md

@@ -15,6 +15,7 @@
 - **A menubar app for Windows.** `windows/` is a Tauri 2 tray app — Rust binary, React popover — that puts today's spend in the notification area and mirrors the macOS menubar screen for screen: agent tabs, period switcher, Trend, Forecast, Pulse, Stats and Plan insights, activity and model breakdowns, optimize findings, CSV/JSON export, launch at login, currency, and theme. Windows has no menubar title, so the number lives in a second tray icon rendered from the system font at the panel's native icon size (Settings can turn it off; the tooltip always carries it). It reads everything through the CLI like the macOS and GNOME clients do, and gates on **codeburn 0.9.9 or newer** — the first release accepting `status --format menubar-json --no-optimize` — showing a setup screen with the install command until it finds one. Refresh follows popover visibility the way the macOS app does: 60 s with optimize findings while open, 2 minutes for today's total while closed, and immediately on open when what you are looking at has gone stale. The Claude quota view never spends Claude's single-use refresh token; on a 401 it re-reads Claude Code's own credential file for a token it has already rotated, matching the macOS client. Ships as an unsigned `.msi` from the `windows-v*` tag, which `codeburn menubar` now installs for you. The same crate still builds and runs a tray on Linux, but that stays experimental and unreleased — `gnome/` is the supported Linux surface.
 
 ### Added (CLI)
+- **Copilot input/cache tokens are read per request from `~/.copilot/session-store.db`.** Previously, codeburn relied on `session.shutdown` rollups from the Copilot CLI and GitHub Copilot desktop app. Those rollups are written only after a clean shutdown, stamp all usage on the shutdown day, and reset their counters at in-session compaction — so a crash could lose an entire session's input/cache usage, and even cleanly-closed long sessions were silently truncated. On one machine with long history, reading the per-request rows recovered about 35% of actual Copilot spend. Covered sessions now use per-request tokens with their real timestamps, counted exactly once against existing rollups and never added as extra calls or turns. Pre-store CLI sessions continue using the unchanged rollup path, and a locked or unreadable store defers only its own re-read instead of prematurely sealing daily history. Copilot reasoning tokens are also no longer double-billed: they are a subset of output already priced through the per-turn calls. This triggers a one-time re-parse, with the daily cache bumped to v21 to re-derive finalized days. Because that reconciliation keeps changing while a session is live, `codeburn sync push` holds a Copilot session back until it has been quiet for 24 hours and then sends it once, final — the sent-ledger is append-once, so a value sent mid-reconciliation could never be corrected at the receiver (#988). Sync also pins each Copilot session to whichever of the two shapes it was first synced in — the `session.shutdown` rollup, or per-request rows plus a residual — because a usage span cannot be retracted and sending the same tokens in the other shape would double them at the receiver permanently. That holds in both directions: a session synced before this release keeps its rollup and never sends rows, and a session synced as rows never sends the rollup that starts serving again once the 90-day age-out prunes them. `--dry-run` reports the frozen count, and `codeburn sync reset --confirm` re-pushes everything under the new breakdown for anyone who can clear the receiver too. (#946)
 - **DeepSeek Harness (`dsh`) is now a supported provider.** Reads DeepSeek's open-source agent harness from `~/.dsh/sessions` (`DSH_HOME` relocates the root), both the default zstd logs and the uncompressed `session.jsonl` variant. A `.zstd` log is a concatenation of independent zstd frames, one per write batch, so it is decoded frame by frame behind a structural frame scan and a torn trailing frame from a crashed writer is ignored rather than failing the file (needs Node 22.15+ for `zlib` zstd; below that dsh is skipped with a notice instead of counted as $0). One call per `(turn, step)`, with the step's final `assistant/message` usage superseding the streamed `assistant/chunk` sample of the same call rather than adding to it, the model taken from the message that served the step, and reasoning tokens billed at the output rate. DSH records tokens but no cost, so calls are priced from the shared tables. The events a forked session replays from its parent are skipped, since codeburn already counts the parent's own log. The session format is pinned at version 0 upstream with no compatibility implied, so a log stamped with any other version is skipped with a notice instead of read under today's assumptions.
 
 ### Changed

docs/providers/copilot.md

@@ -8,15 +8,17 @@ GitHub Copilot Chat (CLI, VS Code core chat sessions, VS Code extension transcri
 
 ## Where it reads from
 
-Three JSONL locations plus an optional OpenTelemetry SQLite source (see below). OTel is
-preferred when present; chatSessions are only discovered when no OTel source is found.
-Other discovered sources are walked on every run; results merge and dedupe.
+Three JSONL locations plus two optional SQLite sources (see the OTel and session-store
+sections). OTel is preferred when present; chatSessions are only discovered when no OTel
+source is found. Other discovered sources are walked on every run; results merge and
+dedupe.
 
 1. **Legacy CLI sessions:** `~/.copilot/session-state/`
 2. **VS Code core chat sessions:** `~/Library/Application Support/Code/User/workspaceStorage/<hash>/chatSessions/*.jsonl` plus `~/Library/Application Support/Code/User/globalStorage/emptyWindowChatSessions/*.jsonl` and equivalents on Windows / Linux
 3. **VS Code transcripts:** `~/Library/Application Support/Code/User/workspaceStorage/<hash>/GitHub.copilot-chat/transcripts/` and equivalents on Windows / Linux
 4. **OTel SQLite store:** VS Code Copilot Chat's `agent-traces.db` (see the OTel section). Preferred when present because it carries full input / output / cache token counts; legacy JSONL sources only record output tokens.
-5. **JetBrains IDE sessions:** `~/.config/github-copilot/<ide>/<kind>/<storeId>/copilot-*-nitrite.db` (see the JetBrains section). Covers IntelliJ IDEA, PyCharm, RubyMine, etc.
+5. **CLI session store:** `~/.copilot/session-store.db` (see the session-store section). One `assistant_usage_events` row per API request — the authoritative input/cache source for CLI and GitHub desktop-app sessions.
+6. **JetBrains IDE sessions:** `~/.config/github-copilot/<ide>/<kind>/<storeId>/copilot-*-nitrite.db` (see the JetBrains section). Covers IntelliJ IDEA, PyCharm, RubyMine, etc.
 
 ## Storage format
 
@@ -46,6 +48,100 @@ instead of trying to dedupe across stores.
   before the upgrade cannot be recovered, so monotonicity starts from the upgrade point,
   not retroactively.
 
+## Session store (CLI / GitHub desktop app)
+
+The Copilot CLI and the GitHub Copilot desktop app both write
+`~/.copilot/session-store.db` (SQLite/WAL) unconditionally. Its
+`assistant_usage_events` table records one row per API request as it happens —
+where the `session.shutdown` rollup in `events.jsonl` is written only on clean
+shutdown (a crash loses the leg's input/cache accounting), lumps each leg into
+one per-model total, and resets its counters at in-session compaction. Rows are
+therefore authoritative for input / cache-read / cache-write / reasoning
+tokens, with real per-request timestamps; per-turn output stays owned by the
+`events.jsonl` `assistant.message` calls. `input_tokens` is cache-INCLUSIVE
+(input + cache_read + cache_write), the same convention as the rollups; the
+parser emits the uncached remainder. Override the path with
+`CODEBURN_COPILOT_SESSION_STORE_DB` (deliberately NOT in the env fingerprint —
+see the #927 ruling in `src/session-cache.ts`).
+
+- **Rollup reconciliation happens at serve time, per (session, model), in
+  `parseProviderSources`** — never in the parser. Both representations always
+  parse and cache; wherever store rows exist for a pair, the rollup calls are
+  dropped and each rollup leg's usage beyond the rows in its own interval
+  serves once as a synthesized residual call at that leg's timestamp. Sessions
+  with no rows (pre-store CLI builds) keep the rollup path unchanged.
+- **A leg's interval starts at its last successful compaction, not at the
+  previous leg.** In-session compaction resets the CLI's rollup counters, so a
+  leg containing one describes only its post-compaction requests. Running the
+  subtraction from the previous leg would cancel that leg's usage against the
+  whole pre-compaction conversation and leave the residual short by exactly
+  that much — invisible while the store is complete (the floor hides it),
+  permanent once a partial snapshot is sealed into a day. Pre-compaction rows
+  still serve; they just stop cancelling usage the rollup never claimed.
+  Recognized events, from `@github/copilot` 1.0.80: `session.compaction_start`
+  (nothing needed from it) and `session.compaction_complete`, whose `success`
+  is read and every other field ignored. Only `success: true` anchors; a failed
+  or absent compaction falls back to the previous-leg interval. Multiple
+  compactions in one leg: the last wins. The stamp rides on the cached rollup
+  call as `compactedAt`.
+  *Accepted, bounded over-serve:* the summarization call has its own usage
+  (`compactionTokensUsed`, charged through the CLI's ordinary `recordUsage`).
+  If it also writes an `assistant_usage_events` row — likely, since it carries
+  the same `copilotUsage.totalNanoAiu` envelope every request does, but the
+  writer is native and not verifiable from the JS bundle — that row sits just
+  before the compaction stamp and so falls outside the interval, while the
+  post-reset rollup counts it. One summarization request per compaction can
+  therefore serve twice. Deliberately not subtracted from the payload: the
+  triggering request completes immediately before the compaction too, so any
+  grace window wide enough to catch the summarization row also catches a real
+  request and turns an over-serve into a loss. Resolvable by anyone with a real
+  store: check whether a row exists at the compaction stamp.
+- **Behavioral weight.** Rollups and residuals are aggregate accounting: tokens
+  and cost count, but never api-call / model-call / turn weight. A store row
+  pairs with its per-turn call by timestamp adjacency (2-minute window,
+  computed over the full serve set); only unpaired rows — crash-recovered,
+  store-only requests — count as calls.
+- **Failure semantics.** True absence (ENOENT, no sqlite driver, `no such
+  table/column` from pre-store CLI builds) reads as absent — no source, rollups
+  rule. Every other failure (locked, EACCES, corrupt, mid-replace) emits the
+  source anyway: its parse defers on the busy shape, previously cached rows
+  keep serving, and the pass reports incomplete hydration so the daily
+  backfill holds its watermark.
+- **Read order.** The store is parsed AFTER every other copilot source in the
+  same pass, and a store served from cache that moves during the pass marks
+  the pass incomplete. Rows commit strictly before their leg's `session.shutdown`
+  line, so reading the store last makes the row set a superset of anything a
+  rollup we read can claim — a session that shuts down mid-pass can no longer
+  leave a leg reconciled against rows that had not landed. What is left is the
+  harmless direction: a row whose journal partner has not arrived yet serves
+  its own tokens and pairs on the next pass.
+- **Durable cache.** Rides the same `durableSources` union as OTel: still-
+  discovered sources are never aged out, and orphans age out at 90 days. A
+  deleted store's rows serve as orphans until then. Reconciliation reads only
+  cached contents, so deleting or resetting the store never changes served
+  totals. A parse-version bump carries every cached entry forward and re-reads
+  the live source into it, because a DB that still exists is not a DB that can
+  still re-derive its pruned rows.
+- **Billing metadata.** Each row's `total_nano_aiu` and `request_multiplier`
+  are captured onto the cached call when the store's schema has them (older
+  stores parse identically without). Nothing prices or displays them yet —
+  billing-grade cost is upstream #890.
+- **Sync.** `codeburn sync push` holds a copilot session until it has been
+  quiet for 24 hours. The reconciliation output is mutable (a residual shrinks
+  as rows land, a rollup is dropped once rows cover its leg, a row's pairing
+  flips), and the sent-ledger is append-once, so a value sent mid-reconciliation
+  could never be corrected at the receiver (#988). Nothing is dropped; the next
+  push after the window sends it once, final. Separately, a session is frozen
+  into whichever of the two shapes it was FIRST synced in — the raw rollup, or
+  rows plus residuals — because the receiver cannot be told to drop what it
+  already holds. Both directions matter: a session synced by a pre-store
+  version never sends rows, and a session synced as rows never sends the
+  rollup that starts serving again once the 90-day age-out prunes them. That is
+  a bounded under-count at the receiver in place of an unbounded over-count;
+  `codeburn sync reset --confirm` re-pushes everything under the new breakdown
+  for anyone who can clear the receiver too.
+- **Requires Node 22+** (`node:sqlite`), same as the OTel source.
+
 ## JetBrains IDEs (IntelliJ, PyCharm, …)
 
 The JetBrains Copilot plugin does **not** write to any of the VS Code or CLI
@@ -161,11 +257,11 @@ surfaces, add a reader with a captured fixture.)
 
 ## Caching
 
-None for the JSONL sources. The OTel source uses a durable cache (see above).
+None for the JSONL sources. The OTel and session-store sources use the durable cache (see above).
 
 ## Deduplication
 
-Legacy JSONL and transcript sessions dedupe per `messageId`. Core chat sessions dedupe per `copilot-chatsession:<sessionId>:<requestId>`, and are not discovered when an OTel source is present. JetBrains `.db` turns dedupe per `copilot:jb:<conversationId>:<turnIndex>` (a per-conversation index, plus reply-content dedup within each conversation). These sources otherwise touch disjoint locations from the VS Code / CLI sources.
+Legacy JSONL and transcript sessions dedupe per `messageId`. Core chat sessions dedupe per `copilot-chatsession:<sessionId>:<requestId>`, and are not discovered when an OTel source is present. Session-store rows dedupe per `copilot-store:<sessionId>:<rowId>:<hash>` (the hash covers `created_at`, token counts, and model, so a same-path DB reset reusing AUTOINCREMENT ids cannot alias a different request onto a cached key); shutdown rollups per `copilot:<sessionId>:shutdown:<model>:<n>`, with serve-time residuals synthesized (never cached) under `copilot:<sessionId>:shutdown-residual:<model>:<leg>`. JetBrains `.db` turns dedupe per `copilot:jb:<conversationId>:<turnIndex>` (a per-conversation index, plus reply-content dedup within each conversation). These sources otherwise touch disjoint locations from the VS Code / CLI sources.
 
 If a workspace hash contains at least one `chatSessions/*.jsonl` file, the provider skips that hash's legacy `GitHub.copilot-chat/transcripts/` directory. The core chat session journal is the modern token-bearing source for the same conversations, so reading both would inflate call counts.
 
@@ -180,6 +276,31 @@ Copilot does not always tag the model on each message. The parser infers it from
 
 See `copilot.ts:176-213`.
 
+## Sharp edges
+
+- **A day sealed on a short store snapshot stays short.** Reconciliation
+  converges — the residual retires as rows land — but the daily cache seals a
+  finalized day once and only re-derives it on a version bump. The two
+  realizable ways to seal a short snapshot are both closed: a session that
+  shuts down mid-pass (the store is read after every journal, and a
+  cache-served store that moves mid-pass holds the watermark) and a compacted
+  leg (its interval now starts at the compaction). What remains is a store that
+  is behind its journal with no local signal at all — rows pruned, a restore,
+  a snapshot taken by something outside this process. There is no detector for
+  it: `rollup > rows-in-interval` is exactly the shape of legitimate partial
+  coverage (a store adopted mid-session has it permanently), so fencing on it
+  would hold the watermark forever. Stated plainly: that day is a permanent
+  under-report and the watermark advances past it. It is not a stall, and the
+  next parse's larger derivation cannot reach back to fix it.
+- **A compaction's own summarization call may serve twice.** See the interval
+  anchor above; bounded at one request per compaction, direction is over-serve.
+- **Pairing is ambiguous inside the 2-minute window.** A crash-only row within
+  two minutes of a request whose own row is missing can pair against it,
+  under-counting `apiCalls` by one. Tokens stay exact.
+- **A model string that cannot match its rollup's** (the empty-model `unknown`
+  key) is invisible to per-model reconciliation, so both representations serve:
+  over-serve, never lose.
+
 ## Quirks
 
 - `toolRequests` can be missing or non-array on older sessions; the parser guards against that (`copilot.ts:126`, `:260`).

docs/sync/README.md

@@ -153,6 +153,12 @@ A: No. You run `codeburn sync push` when you want. A future version may offer op
 **Q: What if I push the same data twice?**
 A: Safe. A local sent-ledger tracks what's been sent. Re-pushing the same window doesn't create duplicates.
 
+**Q: Why is today's Copilot usage missing from my dashboard?**
+A: Copilot input/cache usage is reconciled locally between the per-request `session-store.db` rows and the `session.shutdown` rollups, and that reconciliation keeps changing while a session is live — a rollup residual shrinks as the rows covering it land, and a row that looked like a crash-only request becomes supplementary once its journal entry appears. The sent-ledger is append-once, so a value sent mid-reconciliation could never be corrected at the receiver. A Copilot session is therefore held back until it has been quiet for 24 hours, then pushed once, final. Nothing is dropped; `--dry-run` reports how many calls are held.
+
+**Q: Why didn't my old Copilot sessions resync with the new per-request breakdown?**
+A: On purpose. A Copilot session's input/cache can leave your machine in one of two shapes — as one `session.shutdown` **rollup** span per (session, model), or **reconciled** into one span per API request plus a residual for whatever the rows don't cover. They describe the same tokens, so the receiver must never hold both, and a usage span cannot be retracted once the append-once ledger has sent it. Whichever shape a session was first synced in, it stays in; the other is frozen for that session permanently. That runs in both directions: a session synced by a pre-store version keeps its rollup and never sends rows, and a session synced as rows never sends a rollup later — which matters because at the 90-day durable age-out the cached rows are pruned and the rollup starts serving again under a key that was never sent. Growth within the shape a session already uses is unaffected, and per-turn output spans are never frozen. `--dry-run` reports the frozen count. `codeburn sync reset --confirm` clears the local ledger and re-pushes everything under the new breakdown — only do that if the receiver's copy is cleared too, or you get exactly the doubling this avoids.
+
 **Q: What if I'm offline for a week?**
 A: Next push catches up. The default window is 7 days; use `--since 30d` or `--since all` (up to 6 months) for longer gaps. A push runs to completion regardless of size — server rate limits (429) are waited out automatically.