perf(export): serialize HTTP exports with bounded event buffering

[0xbrayo]

HTTP exports currently hold every event and then a complete JSON string in memory. Serialize event rows incrementally through the existing datastore worker into a private temporary file, then serve that file as the response body. This preserves visibility of acknowledged, uncommitted writes and allows database/serialization/write failures to return an HTTP error before a download begins.

Both single-bucket and all-bucket exports retain the JSON format and download naming. The export query explicitly uses the ordered starttime index, avoiding an unbounded sort even if additional indexes exist.

Part of #671 (HTTP export memory). ActivityWatch/aw-android#229 fixes WebView download saving; this PR addresses the separate Rust HTTP buffering path.

Validation:

• cargo test -p aw-datastore --lib and cargo test -p aw-server: export serialization/error tests and all server/API tests pass.
• Regression coverage compares JSON with the previous materialized format, including empty buckets, tied timestamps, clipping, nested/Unicode payloads, corrupt rows, writer failure, missing buckets, response headers, and acknowledged writes before commit.
• Added export_memory, a reproducible example using only synthetic in-memory data. On 100,000 events with 256-character titles, /usr/bin/time -l measured peak RSS of 203,177,984 bytes for materialization versus 54,788,096 bytes for incremental serialization. This includes the in-memory input database and excludes temporary-file I/O; it measures the serialization memory reduction, not HTTP throughput.
• No live database or running server was used for tests or profiling.

Tradeoffs: the completed JSON export occupies temporary disk space (plaintext, with private-file permissions) until the response is dropped. The datastore worker remains occupied while the snapshot is serialized; this PR does not introduce parallel readers or change the consistency contract discussed in #646.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no concrete correctness, security, or repository-rule violation remains.

Summary

• Event rows are queried and serialized individually through the datastore worker.
• Completed files are rewound and streamed as JSON responses while preserving export naming.
• Single-bucket and all-bucket endpoints share the new export responder.
• Regression tests cover pending writes, missing buckets, headers, serialization parity, corrupt rows, and writer failures.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[HTTP export request] --> B[Create private tempfile]
  B --> C[Send Export command to datastore worker]
  C --> D[Query bucket events using ordered index]
  D --> E[Serialize rows incrementally]
  E --> F[Flush and return completed file]
  F --> G[Seek to beginning]
  G --> H[Stream JSON response]

Loading

_{Reviews (1) · Last reviewed commit: "perf(export): serialize HTTP exports wit..."}

[codecov]

Codecov Report

❌ Patch coverage is 91.22807% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.22%. Comparing base (656f3c9) to head (d1ad337).
⚠️ Report is 103 commits behind head on master.

Files with missing lines	Patch %	Lines
aw-datastore/src/export.rs	87.23%	6 Missing ⚠️
aw-datastore/src/worker.rs	81.81%	2 Missing ⚠️
aw-server/src/endpoints/util.rs	85.71%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #677      +/-   ##
==========================================
+ Coverage   70.81%   80.22%   +9.40%     
==========================================
  Files          51       68      +17     
  Lines        2916     6007    +3091     
==========================================
+ Hits         2065     4819    +2754     
- Misses        851     1188     +337     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.