Repository navigation
fix(export): stream large JSON downloads to a cache file - #304
Conversation
Small-bucket saves from ActivityWatch#229 still buffer the payload as a JS string. For ~500k events that hits the 30s axios timeout and OOMs the WebView. - Intercept /export XHRs and call Android.exportFromUrl - Fetch with auth on a background thread into a cache file - Open the existing Save-to picker from that file CSV of a huge bucket still loads events in JS. Fixes ActivityWatch#228 Git-Session-Id: a180614b-5a5a-5d29-84d8-e1c4e076b990
Review guide (auto-posted)Structured warm-up for reviewers — what changed, what to run, where to look. Key files
Suggested verification
Known risks / watch points
Suggested review focus
Generated by |
|
JavaScript-interface callbacks run on the WebView bridge thread, so reading webView.url there can throw before the native download starts. A detached view also drops view?.post, leaking the streamed cache file. Post both the URL resolve and the fetch completion through the main-looper Handler; enqueue if the fragment is still added, otherwise delete the file. Git-Session-Id: a434bdbc-0d89-5467-a891-085a8060b1c3
|
@greptileai review |
|
Erik merged this at 2026-09-23T01:08:14Z as 6484f5c. CI was green (Test, Test E2E 29, Greptile 5/5). All review threads resolved. No further action on this PR. Companion stack also landed: ActivityWatch/aw-server-rust#721, ActivityWatch/aw-webui#993. #228 stays open until Export-all JSON is confirmed on a ~500k-event device; CSV of a huge bucket is still JS-bound. |
When the system file picker is unavailable, shareExport read the whole cached export into a String and wrote it back out on the UI thread, undoing the streaming the main save path gained in ActivityWatch#304. Copy the cache file as a stream on a background thread, then open the share sheet. The now-unused PendingExport.readContent() is removed.
* chore(submodule): bump aw-server-rust to 05c7b38 Picks up ActivityWatch/aw-server-rust#721: send HTTP headers before serializing large exports, so "Export all buckets as JSON" on ~500k events opens the connection during processing instead of waiting for the full body. Native Android fetch (#304) already streams the body to a cache file; without this bump it still blocks on headers until serialization finishes. Also picks up: - ActivityWatch/aw-server-rust#718 — aw-sync dedupe CLI (JNI path unused) - ActivityWatch/aw-server-rust#719 — aw-sync v2 writer, feature-flagged - ActivityWatch/aw-server-rust#720 — install-ndk.sh macOS readlink - aw-webui bump to #988 (not #993) #228 Git-Session-Id: ba128b8a-fc85-51b5-90c0-4c7a4beaed85 * fix(test): skip sanitized-hostname rewrite in upgrade-with-history The test already set hasMigratedHostname, but startup uses the distinct sanitizedHostnameMigratedTo flag. Without it, Java opens the 17MB seeded sqlite.db while the rust worker switches journal_mode to WAL, hits SQLITE_BUSY, panics, and leaves events/count returning 500. Git-Session-Id: 523c0c0e-38a3-5992-bcb3-bf6bf0d76080 * chore(ci): re-trigger E2E after emulator flake on rotation test Git-Session-Id: 7e8e * fix(test): close seed statement so rust can exclusive-lock sqlite.db UpgradeWithHistoryTest left the compiled INSERT statement open. SQLiteStatement holds a SQLiteClosable ref, so db.close() did not drop the last reference and the Java connection stayed alive. rust then panicked on the v6 exclusive index migration (`database is locked`) and events/count stayed 500 for the rest of the 60s wait. Also delete leftover -wal/-shm after close. Git-Session-Id: 858a0c7e-764d-5961-a03a-156ec4922b8d * chore(ci): re-trigger E2E after emulator flake on syncToggleReceivesRealTap Git-Session-Id: 702c * chore(submodule): bump aw-server-rust to ae38664 Git-Session-Id: a2b5 * chore(submodule): bump aw-server-rust to 9008520 (incl. aw-webui#997 streaming CSV) Git-Session-Id: 888e * chore(submodule): bump aw-server-rust to c535d77 (master, 2026-10-06) Carries 14 merged server fixes since 9008520, including the idempotent v3->v4 key_value migration (#792), stopwatch commit-before-ack (#743), in-place re-import (#766), and negative-duration rejection. Git-Session-Id: fc35e8e9-b0fa-4a6e-ba93-9b6c85e93881
Why
#229 saved small-bucket blob downloads, but "Export all" / a huge bucket still dies silently on 0.14.2b4. The web UI waits for a fully-buffered JSON body (30s axios timeout +
JSON.parsein the WebView).Fixes #228
Companion PRs:
Change
/exportXHRs in the injected hook and callAndroid.exportFromUrlCSV of a huge bucket still materializes events in JS.
Tests
./gradlew :mobile:testStandardDebugUnitTest --tests net.activitywatch.android.fragments.WebUIFragmentTest