Skip to content

fix(dexie-cloud): route all blob writebacks through BlobSavingQueue to avoid PSD context loss#2302

Open
liz709 wants to merge 9 commits into
masterfrom
liz/fix-blobsave-hooks-psd-context
Open

fix(dexie-cloud): route all blob writebacks through BlobSavingQueue to avoid PSD context loss#2302
liz709 wants to merge 9 commits into
masterfrom
liz/fix-blobsave-hooks-psd-context

Conversation

@liz709
Copy link
Copy Markdown
Contributor

@liz709 liz709 commented May 14, 2026

Problem

When using Dexie hooks (e.g. Table.hook('creating')) together with lazy blob mode, reads would crash with:

[dexie-cloud:blobResolve] Failed to resolve BlobRefs on <table>

Caused by:

Cannot read properties of undefined (reading 'table')

Root cause: After an async native fetch (blob download), Dexie's PSD zone is no longer active — PSD.trans becomes undefined. The previous code called table.mutate() directly for rw transactions, which flowed through HooksMiddleware.mutate(). That reads PSD.trans to get the current transaction and crashes.

Fix

Remove the direct table.mutate() path entirely. All blob writebacks now go through BlobSavingQueue.saveBlobs(), which:

  1. Uses setTimeout(fn, 0) to run in a fresh JS task, completely outside any PSD context
  2. Opens a proper Dexie rw transaction — always safe regardless of calling context

The BlobSavingQueue path already existed for readonly transactions and was already correctly armored with disableChangeTracking, disableAccessControl, and disableBlobResolve flags.

Verified

  • Build passes (pnpm build clean)
  • Unit tests pass (114/114)
  • Repro script confirms the underlying PSD bug is real; fix avoids the broken path entirely

Fixes lazy blob mode crash reported by @sarink in Discord.

Summary by CodeRabbit

  • Refactor
    • Unified blob download and persistence flow to improve reliability, avoid duplicate downloads, and ensure resolved blobs are persisted in a single, managed queue.
    • Downloader now bounds concurrency and processes work in controlled chunks for steadier performance.
  • Bug Fixes
    • Prevents in-flight reference/cache leaks and ensures pending blob saves can be awaited before completion signals.

Review Change Stack

…o avoid PSD context loss

After an async native fetch (blob download), Dexie's PSD zone is no
longer active, so PSD.trans is undefined. The previous code called
table.mutate() directly for rw transactions, which flowed through
HooksMiddleware.mutate() — that reads PSD.trans and crashes:
  'Cannot read properties of undefined (reading 'table')'
This surfaced as '[dexie-cloud:blobResolve] Failed to resolve BlobRefs on <table>'.

Fix: Remove the direct mutate path entirely. Always route blob writebacks
through BlobSavingQueue.saveBlobs(), which uses setTimeout(fn, 0) to run
in a fresh JS task outside any PSD context, then opens a proper Dexie
rw transaction — always safe regardless of calling context.

Fixes lazy blob mode crash reported when using Dexie hooks with dexie-cloud.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

📝 Walkthrough

Walkthrough

Centralizes blob download lifecycle: middleware now delegates resolved-blob persistence to BlobDownloadTracker.enqueueSave; BlobDownloadTracker adds concurrency slots, owns BlobSavingQueue persistence, and exposes enqueueSave/drainPendingSaves/releaseRefs; eager downloader uses chunked bulkGet to trigger middleware-driven resolve/save.

Changes

Blob download & persistence rework

Layer / File(s) Summary
Middleware: remove BlobSavingQueue wiring and hand off to tracker
addons/dexie-cloud/src/middlewares/blobResolveMiddleware.ts
Middleware drops BlobSavingQueue import/creation, removes queue parameters from cursor/call sites, and always delegates persistence to db.blobDownloadTracker.enqueueSave(...) or calls releaseRefs(...) when no key is present.
BlobDownloadTracker: concurrency, enqueue/release, download internals
addons/dexie-cloud/src/sync/BlobDownloadTracker.ts
Tracker gains MAX_CONCURRENT, slot acquisition/release, retains inFlight until persistence, adds enqueueSave, drainPendingSaves, releaseRefs, and moves network download logic into a private method handling cached token and non-OK responses.
BlobSavingQueue: onPersisted callback and drain semantics
addons/dexie-cloud/src/sync/BlobSavingQueue.ts
Queue constructor accepts onPersisted(refs); drain() waits for queue empty + idle consumer; consumer calls onPersisted(...) after processing items.
Eager downloader: snapshot keys → chunked bulkGet flow
addons/dexie-cloud/src/sync/eagerBlobDownloader.ts
Replaces per-row resolution/updates with snapshot of _hasBlobRefs keys and sequential bulkGet chunks (size derived from MAX_CONCURRENT) to trigger middleware resolve and queued persistence; drains pending saves before completion.
Types & live-query tweak
addons/dexie-cloud/src/types/TXExpandos.ts, src/live-query/live-query.ts
Adds optional mutatedParts?: ObservabilitySet to TXExpandos; switches mutation event registration to globalEvents.storagemutated.subscribe(...).

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • dfahlander

Poem

🐰 I hopped from key to bulk-get land,
Enqueued each blob with careful hand,
Slots kept busy, saves in queue,
Refs cleared tidy, downloads too,
A hop, a munch — persistence grand.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: routing blob writebacks through BlobSavingQueue to fix PSD context loss, which is the core issue addressed across all modified files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch liz/fix-blobsave-hooks-psd-context

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

dfahlander and others added 8 commits May 18, 2026 10:06
…loads of same blob id). No logical code change - but refactor to ease code review
This is not a logical change. Only to make code review less confusing and make sure subscribe/unsubscribe is done using globalEvents.storagemutated instead of the other tedious way for subscribe that was being used before.
Co-authored-by: Copilot <copilot@github.com>
…'t download it again - reuse the promise from the recent download and keep those promises alive until finally persisted.

Needed rearchitecture due to new depencendy between  BlobSavingQueue and BlobDownloadTracker.

Used slightly differently from blobResolveMiddleware and eagerBlobDownloaded.

Co-authored-by: Copilot <copilot@github.com>
… resolve ALL blobs in memory. This could starve RAM.

Fix: We keep relying on blobResolveMiddleware to resolve the blobs, but we set a limit on each query to actually download the blobs in chunks.

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
addons/dexie-cloud/src/sync/eagerBlobDownloader.ts (1)

58-58: ⚡ Quick win

The infinite loop concern is not applicable with the current MAX_CONCURRENT value.

MAX_CONCURRENT is hardcoded to 10 in BlobDownloadTracker.ts, making CHUNK_SIZE equal to 9. The loop at line 104 will advance normally with no risk of infinite loop.

However, if MAX_CONCURRENT becomes configurable in the future (as the comment suggests), consider adding a defensive floor to prevent CHUNK_SIZE from becoming 0:

-const CHUNK_SIZE = MAX_CONCURRENT - 1;
+const CHUNK_SIZE = Math.max(1, MAX_CONCURRENT - 1);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@addons/dexie-cloud/src/sync/eagerBlobDownloader.ts` at line 58, CHUNK_SIZE
currently sets to MAX_CONCURRENT - 1 which can become zero if MAX_CONCURRENT is
ever configured to 1; change the CHUNK_SIZE declaration to enforce a defensive
minimum (e.g. at least 1) so the loop in eagerBlobDownloader (the loop around
line 104) can always advance; update the CHUNK_SIZE computation to use a floor
such as taking the max between 1 and MAX_CONCURRENT - 1, and add a brief comment
referencing MAX_CONCURRENT from BlobDownloadTracker to explain the defensive
guard.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@addons/dexie-cloud/src/sync/eagerBlobDownloader.ts`:
- Line 58: CHUNK_SIZE currently sets to MAX_CONCURRENT - 1 which can become zero
if MAX_CONCURRENT is ever configured to 1; change the CHUNK_SIZE declaration to
enforce a defensive minimum (e.g. at least 1) so the loop in eagerBlobDownloader
(the loop around line 104) can always advance; update the CHUNK_SIZE computation
to use a floor such as taking the max between 1 and MAX_CONCURRENT - 1, and add
a brief comment referencing MAX_CONCURRENT from BlobDownloadTracker to explain
the defensive guard.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 29729273-cf01-48d3-a54a-4c8fc4792f85

📥 Commits

Reviewing files that changed from the base of the PR and between 65f54fb and ed59f5c.

📒 Files selected for processing (6)
  • addons/dexie-cloud/src/middlewares/blobResolveMiddleware.ts
  • addons/dexie-cloud/src/sync/BlobDownloadTracker.ts
  • addons/dexie-cloud/src/sync/BlobSavingQueue.ts
  • addons/dexie-cloud/src/sync/eagerBlobDownloader.ts
  • addons/dexie-cloud/src/types/TXExpandos.ts
  • src/live-query/live-query.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants