feat(editor): add Bear backup import and markdown zip folder hierarchy#14599
feat(editor): add Bear backup import and markdown zip folder hierarchy#14599karl-kaefer wants to merge 14 commits intotoeverything:canaryfrom
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds end-to-end Bear (.bear2bk) import: new BearTransformer parses TextBundle entries from ZIP, converts/cleans markdown (callouts, highlights, tags, title), lazily imports assets into blob store, returns doc IDs plus optional folder hierarchy; updates markdown import return shape, wires UI/i18n, and adjusts HTML highlight delta handling. Changes
sequenceDiagram
autonumber
participant User as "User"
participant UI as "ImportDialog (Frontend)"
participant Transformer as "BearTransformer"
participant JSZip as "JSZip"
participant Parser as "MarkdownAdapter"
participant Blob as "Blob/Asset Store"
participant Doc as "Doc CRUD"
participant Tag as "TagService"
participant Folders as "createFolderStructure"
User->>UI: select .bear2bk and start import
UI->>Transformer: importBearBackup(file, { extensions })
Transformer->>JSZip: load zip & list entries
Transformer->>Transformer: group *.textbundle/ entries
loop per bundle
Transformer->>JSZip: read text.md, info.json, assets/*
Transformer->>Parser: parse markdown (strip metadata, callouts, highlights), extract tags/title
Transformer->>Blob: store assets (sha, mime) -> blob ids
Transformer->>Doc: create document with content and blob refs -> docId
Transformer->>Transformer: collect tags for docId
end
Transformer->>Transformer: build folderHierarchy from tags/paths
Transformer-->>UI: return { docIds, folderHierarchy?, tags }
UI->>Tag: create/match tags and apply to docs
UI->>Folders: create nested folders and link docs
UI-->>User: show import result
🎯 4 (Complex) | ⏱️ ~45 minutes
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 50-97: parseBearTags currently only collects tags from a trailing
footer because the reverse scan breaks on the first non-tag line; to fix,
collect inline tags from the whole document while still preserving removal of a
trailing tag-only footer: run a full forward pass over lines (respecting code
fences using the existing codeFenceState) and call extractTagsFromLine on every
non-code line to accumulate tags, then run the existing reverse scan (or a
separate reverse-only scan) to identify and remove trailing tag-only lines into
tagLineIndices before assembling filteredLines; return deduplicateTags(tags) and
the content as before. Use the existing function names parseBearTags,
extractTagsFromLine, deduplicateTags and the tagLineIndices logic to locate
footer lines to delete.
- Around line 242-251: The replacement callback that builds raw <mark> HTML in
the lines[i].replace call interpolates unescaped content/text (see the anonymous
callback using HIGHLIGHT_COLOR_MAP and firstChar), which allows markup
injection; update the callback to HTML-escape the output before interpolation by
calling a sanitizer like escapeHtml on both the color branch (return `<mark
data-color="${color}">${escapeHtml(text)}</mark>`) and the fallback branch
(return `<mark>${escapeHtml(content)}</mark>`), and add/import a suitable
escapeHtml utility in the module so all emitted mark text is properly escaped.
In `@blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts`:
- Around line 477-513: The code currently treats only paths with parts.length >
2 as hierarchical and drops files when folderParts becomes empty after stripping
a common root; change the initial hierarchy detection to treat parts.length >= 2
as indicating a single-level folder is meaningful, and in the loop preserve
wrapper-root files by not skipping when folderParts.length === 0 — instead, if
folderParts was emptied solely because you stripped a common root (i.e.,
original partsBeforeStrip.length >= 1), restore a single-level folder (use the
original first part or keep a marker) so the file is attached into the
hierarchy; update the logic around hasSubfolders, folderParts, and the early
continue in the for loop (referencing entries, hasSubfolders, folderParts,
partsBeforeStrip, and the FolderHierarchy root/createFolderStructure code paths)
to ensure single-level folders like "docs/file.md" and wrapper-root files like
"root/file.md" are preserved and linked.
In `@packages/frontend/core/src/desktop/dialogs/import/index.tsx`:
- Around line 495-497: The file picker filter for the Bear importer uses
fileOptions.acceptType = 'Zip', which only allows .zip and therefore prevents
selecting .bear2bk files even though BearTransformer.importBearBackup can handle
them; update the acceptType mapping to include '.bear2bk' for the 'Zip'
acceptType (or add a new acceptType e.g. 'Bear' that maps to ['.zip',
'.bear2bk']) and then update the Bear importer configuration (where
fileOptions.acceptType is set) to use the new mapping so the dialog allows
selecting .bear2bk files while keeping existing .zip support.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 80859bda-bef5-49cc-8d88-7eb08cb42bc5
📒 Files selected for processing (8)
blocksuite/affine/inlines/preset/src/adapters/html/html-inline.tsblocksuite/affine/widgets/linked-doc/package.jsonblocksuite/affine/widgets/linked-doc/src/transformers/bear.tsblocksuite/affine/widgets/linked-doc/src/transformers/index.tsblocksuite/affine/widgets/linked-doc/src/transformers/markdown.tsblocksuite/playground/apps/_common/components/starter-debug-menu.tspackages/frontend/core/src/desktop/dialogs/import/index.tsxpackages/frontend/i18n/src/resources/en.json
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (1)
434-447: Use the actual markdown file path forfilePathMiddleware.The bundle scan accepts both
text.mdandtext.txtviaentry.markdownPath, butfullPathis hardcoded as.../text.md. While the image resolution uses only the directory component, this is inconsistent withmarkdown.tsandhtml.ts, which pass the actual file path. Use the stored path to reflect what was actually imported.Suggested change
- const fullPath = `${entry.bundlePath}/text.md`; + const fullPath = entry.markdownPath!;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 434 - 447, The fullPath is hardcoded to `${entry.bundlePath}/text.md` but should use the actual imported file path (entry.markdownPath) so filePathMiddleware receives the real markdown filename; update the Transformer construction to build fullPath from entry.markdownPath (instead of "text.md") before passing it to filePathMiddleware so behavior matches markdown.ts and html.ts and image/file resolution uses the correct stored path.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 460-480: Wrap the per-bundle work (the call to mdAdapter.toDoc and
the subsequent meta patch + collection.meta.setDocMeta) in a try/catch so one
failing bundle won’t abort the whole import; if mdAdapter.toDoc throws, catch
it, log/record the failure and continue without pushing to docIds; if setDocMeta
(or any later step) throws after a doc was returned, attempt to remove the
partially-created doc via the collection API (e.g., call the collection's
delete/remove method for doc.id), then catch and log that removal error and
continue to the next bundle; ensure docIds is only appended to on fully
successful bundle completion.
- Around line 233-244: The convertGfmCallouts function currently runs across the
whole document and rewrites callout markers inside fenced code blocks; modify
convertGfmCallouts to process the markdown line-by-line, track fenced code
blocks with an inCodeBlock boolean toggle (flip when encountering lines that
start a/stop a fence like ```), skip replacement when inCodeBlock is true, and
only apply the existing regex replacement (/^(>\s*)\[!(\w+)\]/) per line when
not in a code block; reference the convertGfmCallouts function and
GFM_CALLOUT_MAP when implementing the line loop and inCodeBlock guard so literal
fenced examples like "> [!NOTE]" are left untouched.
---
Nitpick comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 434-447: The fullPath is hardcoded to
`${entry.bundlePath}/text.md` but should use the actual imported file path
(entry.markdownPath) so filePathMiddleware receives the real markdown filename;
update the Transformer construction to build fullPath from entry.markdownPath
(instead of "text.md") before passing it to filePathMiddleware so behavior
matches markdown.ts and html.ts and image/file resolution uses the correct
stored path.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9410f568-d459-4d27-87fa-a52727b3a79e
📒 Files selected for processing (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## canary #14599 +/- ##
==========================================
- Coverage 57.32% 56.69% -0.64%
==========================================
Files 3125 3126 +1
Lines 169094 169341 +247
Branches 25008 24933 -75
==========================================
- Hits 96937 96011 -926
- Misses 68944 70107 +1163
- Partials 3213 3223 +10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
d48dd97 to
58e6b98
Compare
58e6b98 to
6f19449
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts (1)
519-550:⚠️ Potential issue | 🟠 MajorDon't let one bad Markdown file fail the whole ZIP after partial writes.
Promise.allwill reject on the first thrown import, but earliertoDoccalls may already have created docs. At that point the caller gets no{ docIds, folderHierarchy }, so the UI cannot finish folder placement and the workspace is left with a partial import. Handle failures per file (allSettled/local try-catch) or roll back docs created before the rejection.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts` around lines 519 - 550, The current Promise.all over markdownBlobs will reject on the first error and lose earlier created docs; change the per-file import inside markdownBlobs.map to handle failures individually (wrap each markdownFile handler in a try/catch or use Promise.allSettled) so that createMarkdownImportJob, bindImportedAssetsToJob, MarkdownAdapter.toDoc, and applyMetaPatch continue for other files even if one fails; ensure you still push successful doc ids into docIds and entries into docPathMap, record/log per-file errors for debugging, and then return { docIds, folderHierarchy } so buildMarkdownZipFolderHierarchy can run on the successfully imported docs.
♻️ Duplicate comments (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (1)
474-505:⚠️ Potential issue | 🟠 MajorThe bundle still isn't atomic after
toDoc()succeeds.
docIds.push(doc.id)runs beforesetDocMeta, and the outercatchonly logs. If any later step throws, the partially created doc remains in the workspace and may already be reported as a success. Move the bookkeeping after the last throwing step and deletedoc.idin the failure path.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 474 - 505, The code currently pushes doc.id into docIds before running subsequent mutating steps (setDocMeta and tagDocMap updates), so if any later step throws the outer catch only logs and the partially-created doc remains; fix this by delaying all bookkeeping (docIds.push, tagDocMap.set/push) until after collection.meta.setDocMeta completes successfully (i.e., after the last potential throwing operation in the try block), and in the catch block ensure you remove/delete the created doc (using doc.id) from the workspace/state so no partial artifact remains; reference the MarkdownAdapter.toDoc result (doc), collection.meta.setDocMeta, docIds, and tagDocMap and remove doc.id on failure (and keep the existing console.warn with entry.bundlePath).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 423-424: The code extracts the file extension into the variable
ext from assetRelPath and then looks up mime via extMimeMap.get(ext) but doesn't
normalize case; update the logic so ext is lowercased before the MIME lookup
(e.g., compute ext = (assetRelPath.split('.').at(-1) ?? '').toLowerCase()) so
extMimeMap.get(ext) returns the correct MIME for extensions like "PNG" or "JPG",
leaving extMimeMap and mime variable names unchanged.
- Around line 136-147: The open-tag regex is ASCII-only and drops/truncates
Unicode tags; update the pattern used where openMatch is defined to accept
Unicode characters by either adding the /u flag and using property escapes
(e.g., /^#([\p{L}\p{N}_][\p{L}\p{N}_\/-]*)(.*)$/u) or switching to a negated
class that forbids whitespace and '#' (e.g., /^#([^\s#][^\s#\/-]*)(.*)$/u or
adjusted to allow '/' as needed); ensure you replace the current
/^#([\w][\w\/-]*)(.*)$/ with one of these Unicode-aware patterns so tags like
`#café` and `#日本` parse correctly.
---
Outside diff comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts`:
- Around line 519-550: The current Promise.all over markdownBlobs will reject on
the first error and lose earlier created docs; change the per-file import inside
markdownBlobs.map to handle failures individually (wrap each markdownFile
handler in a try/catch or use Promise.allSettled) so that
createMarkdownImportJob, bindImportedAssetsToJob, MarkdownAdapter.toDoc, and
applyMetaPatch continue for other files even if one fails; ensure you still push
successful doc ids into docIds and entries into docPathMap, record/log per-file
errors for debugging, and then return { docIds, folderHierarchy } so
buildMarkdownZipFolderHierarchy can run on the successfully imported docs.
---
Duplicate comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 474-505: The code currently pushes doc.id into docIds before
running subsequent mutating steps (setDocMeta and tagDocMap updates), so if any
later step throws the outer catch only logs and the partially-created doc
remains; fix this by delaying all bookkeeping (docIds.push, tagDocMap.set/push)
until after collection.meta.setDocMeta completes successfully (i.e., after the
last potential throwing operation in the try block), and in the catch block
ensure you remove/delete the created doc (using doc.id) from the workspace/state
so no partial artifact remains; reference the MarkdownAdapter.toDoc result
(doc), collection.meta.setDocMeta, docIds, and tagDocMap and remove doc.id on
failure (and keep the existing console.warn with entry.bundlePath).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9e88c9bd-0e02-42be-8fea-718fe5f01c21
📒 Files selected for processing (6)
blocksuite/affine/inlines/preset/src/adapters/html/html-inline.tsblocksuite/affine/widgets/linked-doc/package.jsonblocksuite/affine/widgets/linked-doc/src/transformers/bear.tsblocksuite/affine/widgets/linked-doc/src/transformers/index.tsblocksuite/affine/widgets/linked-doc/src/transformers/markdown.tsblocksuite/playground/apps/_common/components/starter-debug-menu.ts
✅ Files skipped from review due to trivial changes (3)
- blocksuite/affine/widgets/linked-doc/package.json
- blocksuite/playground/apps/_common/components/starter-debug-menu.ts
- blocksuite/affine/widgets/linked-doc/src/transformers/index.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- blocksuite/affine/inlines/preset/src/adapters/html/html-inline.ts
There was a problem hiding this comment.
🧹 Nitpick comments (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (1)
454-458: Consider using the sharedcreateCollectionDocCRUDhelper.The inlined
docCRUDdefinition duplicates the logic increateCollectionDocCRUDfrom markdown.ts. Reusing the helper improves consistency.♻️ Suggested refactor
+import { createCollectionDocCRUD } from './markdown'; + // In importBearBackup: const job = new Transformer({ schema, blobCRUD: collection.blobSync, - docCRUD: { - create: (id: string) => collection.createDoc(id).getStore({ id }), - get: (id: string) => collection.getDoc(id)?.getStore({ id }) ?? null, - delete: (id: string) => collection.removeDoc(id), - }, + docCRUD: createCollectionDocCRUD(collection), middlewares: [🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 454 - 458, The inlined docCRUD duplicates existing logic; replace the manual object with the shared helper by calling createCollectionDocCRUD(collection) (from markdown.ts) instead of manually wiring collection.createDoc, collection.getDoc, and collection.removeDoc so the code uses the common implementation and stays consistent with other modules.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 454-458: The inlined docCRUD duplicates existing logic; replace
the manual object with the shared helper by calling
createCollectionDocCRUD(collection) (from markdown.ts) instead of manually
wiring collection.createDoc, collection.getDoc, and collection.removeDoc so the
code uses the common implementation and stays consistent with other modules.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 24b58f9d-b761-4a34-99a0-a90ec19dd235
📒 Files selected for processing (3)
blocksuite/affine/widgets/linked-doc/package.jsonblocksuite/affine/widgets/linked-doc/src/transformers/bear.tsblocksuite/playground/apps/_common/components/starter-debug-menu.ts
✅ Files skipped from review due to trivial changes (2)
- blocksuite/affine/widgets/linked-doc/package.json
- blocksuite/playground/apps/_common/components/starter-debug-menu.ts
a1ed555 to
d7bc6a2
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (1)
476-507:⚠️ Potential issue | 🟠 MajorRollback the created doc when later bundle steps fail.
After
toDocsucceeds,setDocMeta(...)can still throw, but this catch only warns. BecausedocIds.push(doc.id)already happened, a failed bundle can leave a page behind and return ids that are out of sync withtagDocMap/folderHierarchy. Keep the created id local until the whole bundle succeeds, and remove the doc in the catch path if post-processing fails.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 476 - 507, The code currently pushes doc.id into docIds immediately after mdAdapter.toDoc succeeds but before post-processing (collection.meta.setDocMeta, tagDocMap updates, folderHierarchy changes), so if those later steps throw the doc remains persisted and docIds is inconsistent; change the flow to keep the created id local (e.g., a local variable createdDocId) and only push to docIds and update tagDocMap/folderHierarchy after all post-processing succeeds; in the catch block for the try surrounding mdAdapter.toDoc and subsequent steps, if a createdDocId exists call the removal routine (delete/remove document via the same collection or API used elsewhere) to rollback the partially-created doc, and ensure collection.meta.setDocMeta, tagDocMap updates, and folderHierarchy modifications are performed before mutating docIds so state stays consistent (affecting mdAdapter.toDoc, collection.meta.setDocMeta, tagDocMap, folderHierarchy, and docIds).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 402-405: The code currently skips any note with an empty body via
the check on rawMarkdown ("if (!rawMarkdown.trim()) continue;"); instead,
preserve non-trashed notes with empty content by removing the continue and
explicitly handling empty markdown (e.g., set markdown = '' or a placeholder) so
the note is included in the backup flow; update the logic around rawMarkdown,
entry.markdownPath and the zip.file(...).async(...) call to treat empty strings
as valid note bodies rather than silently dropping them.
- Around line 373-392: After collecting bundles into validBundles, add a check
after the loop to throw a descriptive error when validBundles.length === 0
instead of returning a successful empty result; update the transformer (in this
file's Bear transformer function where validBundles is populated) to throw
something like "No valid Bear .textbundle entries found in ZIP" so arbitrary
ZIPs are rejected, ensuring callers see a failure rather than { docIds: [] } and
the UI does not show a no-op import as success.
---
Duplicate comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 476-507: The code currently pushes doc.id into docIds immediately
after mdAdapter.toDoc succeeds but before post-processing
(collection.meta.setDocMeta, tagDocMap updates, folderHierarchy changes), so if
those later steps throw the doc remains persisted and docIds is inconsistent;
change the flow to keep the created id local (e.g., a local variable
createdDocId) and only push to docIds and update tagDocMap/folderHierarchy after
all post-processing succeeds; in the catch block for the try surrounding
mdAdapter.toDoc and subsequent steps, if a createdDocId exists call the removal
routine (delete/remove document via the same collection or API used elsewhere)
to rollback the partially-created doc, and ensure collection.meta.setDocMeta,
tagDocMap updates, and folderHierarchy modifications are performed before
mutating docIds so state stays consistent (affecting mdAdapter.toDoc,
collection.meta.setDocMeta, tagDocMap, folderHierarchy, and docIds).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 42109dab-00f4-4085-af95-aba021f4191f
⛔ Files ignored due to path filters (1)
yarn.lockis excluded by!**/yarn.lock,!**/*.lock
📒 Files selected for processing (10)
blocksuite/affine/inlines/preset/src/adapters/html/html-inline.tsblocksuite/affine/widgets/linked-doc/package.jsonblocksuite/affine/widgets/linked-doc/src/transformers/bear.tsblocksuite/affine/widgets/linked-doc/src/transformers/index.tsblocksuite/affine/widgets/linked-doc/src/transformers/markdown.tsblocksuite/playground/apps/_common/components/starter-debug-menu.tspackages/frontend/core/src/desktop/dialogs/import/index.tsxpackages/frontend/i18n/src/i18n-completenesses.jsonpackages/frontend/i18n/src/i18n.gen.tspackages/frontend/i18n/src/resources/en.json
✅ Files skipped from review due to trivial changes (6)
- packages/frontend/i18n/src/i18n-completenesses.json
- blocksuite/affine/widgets/linked-doc/src/transformers/index.ts
- blocksuite/affine/widgets/linked-doc/package.json
- blocksuite/affine/inlines/preset/src/adapters/html/html-inline.ts
- packages/frontend/i18n/src/resources/en.json
- blocksuite/playground/apps/_common/components/starter-debug-menu.ts
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (2)
337-366:⚠️ Potential issue | 🟠 MajorReject ZIPs that do not contain Bear notes.
A random ZIP with no markdown-bearing
.textbundleentries still falls through to a successful empty import. Fail fast once the archive scan finishes and no note bundles were discovered; if you want “all notes were trashed” to stay distinct, handle that separately from the archive-format check.🧭 Minimal fix
zip.forEach((path, _entry) => { if (path.includes('__MACOSX') || path.includes('.DS_Store')) return; @@ } }); + + const hasMarkdownBundle = [...bundleMap.values()].some( + entry => entry.markdownPath !== null + ); + if (!hasMarkdownBundle) { + throw new Error('No Bear .textbundle notes found in archive'); + } // Read info.json for all bundles to filter out trashed notesBased on learnings, the Bear picker intentionally uses
acceptType: 'Zip'because the WindowsshowOpenFilePickerAPI silently ignores custom or unknown extensions like.bear2bk, so transformer-side archive validation is important here.Also applies to: 375-394
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 337 - 366, The archive scan (zip.forEach ... building bundleMap) currently allows ZIPs with no .textbundle entries to proceed; after the loop that builds bundleMap check whether any bundles were discovered (e.g., bundleMap.size === 0) and immediately reject/fail the import with a clear error (or throw) indicating the archive is not a Bear export, ensuring this is handled at the transformer level (references: zip.forEach, bundleMap, BundleEntry, bundle.markdownPath/infoJsonPath/assetPaths); keep separate logic elsewhere for the "all notes were trashed" case rather than treating empty bundleMap as a valid import.
404-406:⚠️ Potential issue | 🟠 MajorPreserve empty Bear notes instead of skipping them.
Line 406 drops any non-trashed note whose body is blank, so empty notes disappear from the backup. If
MarkdownAdapterneeds special handling for'', do that explicitly, but this branch should notcontinue.📝 Minimal first step
// Read markdown (decompress on demand) const rawMarkdown = await zip.file(entry.markdownPath!)!.async('string'); - if (!rawMarkdown.trim()) continue; const { tags, content: cleanedMarkdown } = parseBearTags(rawMarkdown);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 404 - 406, The code in bear.ts currently skips any non-trashed note whose rawMarkdown is empty by doing `if (!rawMarkdown.trim()) continue`, which causes empty Bear notes to be dropped from backups; change this to preserve empty bodies: remove the early `continue` and instead handle the empty-string case explicitly where the markdown is processed (e.g., when calling the MarkdownAdapter or downstream transformer) so that an empty string is passed through or converted to the adapter's expected representation. Locate the `rawMarkdown` usage (the `rawMarkdown = await zip.file(entry.markdownPath!)!.async('string')` variable and subsequent logic) and adjust processing so blank content is not skipped but handled appropriately by `MarkdownAdapter` or the note serialization path.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 309-320: extractTitle currently scans every line and can pick up a
literal "# Heading" inside fenced code blocks; update extractTitle to ignore
fenced blocks by tracking an inCodeBlock flag (like other markdown transforms).
Specifically, inside the extractTitle function, detect fence open/close lines
(e.g., lines starting with ``` or ~~~), toggle an inCodeBlock boolean and skip
processing lines while inCodeBlock is true, then only run the /^#\s+(.+)/ match
when not in a fenced block so only real headings are considered.
---
Duplicate comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 337-366: The archive scan (zip.forEach ... building bundleMap)
currently allows ZIPs with no .textbundle entries to proceed; after the loop
that builds bundleMap check whether any bundles were discovered (e.g.,
bundleMap.size === 0) and immediately reject/fail the import with a clear error
(or throw) indicating the archive is not a Bear export, ensuring this is handled
at the transformer level (references: zip.forEach, bundleMap, BundleEntry,
bundle.markdownPath/infoJsonPath/assetPaths); keep separate logic elsewhere for
the "all notes were trashed" case rather than treating empty bundleMap as a
valid import.
- Around line 404-406: The code in bear.ts currently skips any non-trashed note
whose rawMarkdown is empty by doing `if (!rawMarkdown.trim()) continue`, which
causes empty Bear notes to be dropped from backups; change this to preserve
empty bodies: remove the early `continue` and instead handle the empty-string
case explicitly where the markdown is processed (e.g., when calling the
MarkdownAdapter or downstream transformer) so that an empty string is passed
through or converted to the adapter's expected representation. Locate the
`rawMarkdown` usage (the `rawMarkdown = await
zip.file(entry.markdownPath!)!.async('string')` variable and subsequent logic)
and adjust processing so blank content is not skipped but handled appropriately
by `MarkdownAdapter` or the note serialization path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: cc812bf2-6fa1-4b9c-93ab-5aedfef2785e
📒 Files selected for processing (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts
There was a problem hiding this comment.
🧹 Nitpick comments (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (1)
515-517: Consider logging the actual error for easier debugging.The catch block only logs the bundle path, which may make debugging difficult when imports fail silently. Including the error message would help diagnose issues.
🔧 Proposed improvement
- } catch { - console.warn(`Failed to import bundle: ${entry.bundlePath}`); + } catch (err) { + console.warn(`Failed to import bundle: ${entry.bundlePath}`, err); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 515 - 517, The catch block currently swallows the error and only logs entry.bundlePath; update the catch to capture the exception (e.g., catch (err)) and include the error details in the log for easier debugging—replace the existing console.warn in the failing import area with a message that combines the bundle path (entry.bundlePath) and the error object/message (err) so callers see both context and the underlying error.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 515-517: The catch block currently swallows the error and only
logs entry.bundlePath; update the catch to capture the exception (e.g., catch
(err)) and include the error details in the log for easier debugging—replace the
existing console.warn in the failing import area with a message that combines
the bundle path (entry.bundlePath) and the error object/message (err) so callers
see both context and the underlying error.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 5bab11d4-c76f-4985-8591-c95091f375cc
📒 Files selected for processing (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts
There was a problem hiding this comment.
♻️ Duplicate comments (2)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts (2)
493-514:⚠️ Potential issue | 🟠 MajorOnly append
docIdsafter the bundle finishes cleanly.Line 494 records the doc before
setDocMetaand tag bookkeeping run. If one of those later steps throws, the bundle is warned as failed but its ID is still returned, whiletagsandfolderHierarchyomit it. If metadata patch failures are meant to be non-fatal, catch them separately instead of letting the outer bundle catch handle them.🧩 Consistent ordering
if (doc) { - docIds.push(doc.id); - const metaPatch: Record<string, unknown> = {}; if (bearMeta?.creationDate) { const ts = Date.parse(String(bearMeta.creationDate)); if (!isNaN(ts)) metaPatch.createDate = ts; } @@ for (const tag of tags) { if (!tagDocMap.has(tag)) { tagDocMap.set(tag, []); } tagDocMap.get(tag)!.push(doc.id); } + + docIds.push(doc.id); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 493 - 514, The code currently pushes doc.id into docIds before running collection.meta.setDocMeta and tag/folder bookkeeping, so if those later steps fail the bundle is marked failed but the id remains; move the docIds.push(doc.id) to after the meta/tag/folder operations succeed (i.e., after collection.meta.setDocMeta and the tagDocMap/folderHierarchy updates) and only then append the id. Additionally, if metadata patch failures are intended to be non-fatal, wrap collection.meta.setDocMeta in its own try/catch and log or handle the error without throwing to preserve correct bundle success semantics.
66-73:⚠️ Potential issue | 🟠 MajorHandle
~~~fences anywhere you track code blocks.Lines 69, 243, 288, and 315 only recognize backtick fences. A note that uses
~~~code blocks can still have footer tags stripped from code, literal> [!NOTE]/==...==examples rewritten, or a# headinginside code promoted to the doc title.🛠️ Minimal fix
+function isFenceLine(line: string): boolean { + return /^\s*(?:`{3,}|~{3,})/.test(line); +} + function parseBearTags(markdown: string): { tags: string[]; content: string; } { const lines = markdown.split('\n'); @@ const codeFenceState: boolean[] = []; let inCodeBlock = false; for (const line of lines) { - if (line.trimStart().startsWith('```')) { + if (isFenceLine(line)) { inCodeBlock = !inCodeBlock; } codeFenceState.push(inCodeBlock); } @@ function convertGfmCallouts(markdown: string): string { const lines = markdown.split('\n'); let inCodeBlock = false; for (let i = 0; i < lines.length; i++) { - if (lines[i].trimStart().startsWith('```')) { + if (isFenceLine(lines[i])) { inCodeBlock = !inCodeBlock; continue; } @@ function convertHighlights(markdown: string): string { const lines = markdown.split('\n'); let inCodeBlock = false; for (let i = 0; i < lines.length; i++) { - if (lines[i].trimStart().startsWith('```')) { + if (isFenceLine(lines[i])) { inCodeBlock = !inCodeBlock; continue; } @@ function extractTitle(markdown: string, bundleName: string): string { const lines = markdown.split('\n'); let inCodeBlock = false; for (const line of lines) { - if (line.trimStart().startsWith('```')) { + if (isFenceLine(line)) { inCodeBlock = !inCodeBlock; continue; }Also applies to: 243-245, 288-290, 315-317
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts` around lines 66 - 73, The code only toggles code-block state on backtick fences; add a helper isFenceLine(line: string) that returns true for lines whose trimmed start is either "```" or "~~~", then replace the direct checks in the codeFenceState loop and in functions convertGfmCallouts, convertHighlights, and extractTitle (where they call line.trimStart().startsWith('```')) to call isFenceLine(line) so both ``` and ~~~ fences are recognized consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts`:
- Around line 493-514: The code currently pushes doc.id into docIds before
running collection.meta.setDocMeta and tag/folder bookkeeping, so if those later
steps fail the bundle is marked failed but the id remains; move the
docIds.push(doc.id) to after the meta/tag/folder operations succeed (i.e., after
collection.meta.setDocMeta and the tagDocMap/folderHierarchy updates) and only
then append the id. Additionally, if metadata patch failures are intended to be
non-fatal, wrap collection.meta.setDocMeta in its own try/catch and log or
handle the error without throwing to preserve correct bundle success semantics.
- Around line 66-73: The code only toggles code-block state on backtick fences;
add a helper isFenceLine(line: string) that returns true for lines whose trimmed
start is either "```" or "~~~", then replace the direct checks in the
codeFenceState loop and in functions convertGfmCallouts, convertHighlights, and
extractTitle (where they call line.trimStart().startsWith('```')) to call
isFenceLine(line) so both ``` and ~~~ fences are recognized consistently.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b5645e29-5d35-44fe-b12d-92e1cfc4d63c
📒 Files selected for processing (1)
blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts
|
@darkskygit Could you please re-run the failed CI jobs? Both failures appear unrelated to this PR. The share-page E2E test (share-page-1.spec.ts:210) also fails on canary (run 24001775690), and the server config test is a Redis teardown issue. |
f7467d7 to
fe6186f
Compare
|
Sorry for pinging you again @darkskygit - could you please re-trigger? forgot to push the changes. Sorry! |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts (1)
580-588: Consider pre-computing common root to reduce O(n²) to O(n).The
allSameRootcheck inside the loop iterates all entries for each entry, resulting in O(n²) complexity. For large zip archives this could be noticeable. Computing the common root once before the loop would be more efficient.♻️ Proposed optimization
function buildMarkdownZipFolderHierarchy( entries: Array<{ fullPath: string; docId: string }> ): FolderHierarchy | undefined { // Check if any entries have folder structure const hasSubfolders = entries.some(e => { const parts = e.fullPath.split('/').filter(Boolean); // More than just "root/file.md" — need at least one real subfolder return parts.length > 2; }); if (!hasSubfolders && entries.length > 0) { // All files are at the same level, no folder hierarchy needed return undefined; } + // Pre-compute common root once (O(n) instead of O(n²)) + const firstEntry = entries[0]; + const firstRoot = firstEntry + ? firstEntry.fullPath.split('/').filter(Boolean)[0] + : null; + const allShareCommonRoot = + firstRoot != null && + entries.every(e => e.fullPath.startsWith(firstRoot + '/')); + const root: FolderHierarchy = { name: '', path: '', children: new Map(), }; for (const { fullPath, docId } of entries) { const parts = fullPath.split('/').filter(Boolean); const fileName = parts.pop(); // Remove filename if (!fileName) continue; - // Skip the root zip directory (first part) if all entries share it let folderParts = parts; - if (folderParts.length > 0) { - // Check if first part is a common root directory — skip it - const allSameRoot = entries.every(e => - e.fullPath.startsWith(folderParts[0] + '/') - ); - if (allSameRoot) { - folderParts = folderParts.slice(1); - } + // Skip the common root directory if all entries share it + if (allShareCommonRoot && folderParts.length > 0) { + folderParts = folderParts.slice(1); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts` around lines 580 - 588, The allSameRoot check currently recomputes startsWith over entries for each entry (causing O(n²)); move this work outside the per-entry logic by computing the common root once: derive the candidate root from folderParts[0] (or from the first entry) and evaluate a single allSameRoot boolean by scanning entries once (e.g., entries.every(e => e.fullPath.startsWith(candidateRoot + '/'))), then if true slice folderParts = folderParts.slice(1); update the code locations where allSameRoot, folderParts and entries are referenced (the variables named folderParts, entries and the allSameRoot check) to use this precomputed value so the loop becomes O(n).packages/frontend/core/src/desktop/dialogs/import/index.tsx (1)
636-661: Extract folder hierarchy application into a shared helper.The folder hierarchy creation pattern is repeated nearly identically in
markdownZip(lines 402-425),notion(lines 484-511), andbear(lines 636-661). Consider extracting into a helper function to reduce duplication.♻️ Proposed helper function
function applyFolderHierarchy( organizeService: OrganizeService, folderHierarchy: FolderHierarchy, explorerIconService?: ExplorerIconService ): string | undefined { if (folderHierarchy.children.size === 0) { return undefined; } try { const { folderId, docLinks } = createFolderStructure( organizeService, folderHierarchy, null, explorerIconService ); for (const { folderId, docId } of docLinks) { const folder = organizeService.folderTree.folderNode$(folderId).value; if (folder) { const index = folder.indexAt('after'); folder.createLink('doc', docId, index); } } return folderId || undefined; } catch (error) { logger.warn('Failed to create folder structure:', error); return undefined; } }Then each import function can call:
const rootFolderId = folderHierarchy && organizeService ? applyFolderHierarchy(organizeService, folderHierarchy, explorerIconService) : undefined;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/frontend/core/src/desktop/dialogs/import/index.tsx` around lines 636 - 661, The repeated folder-hierarchy creation logic should be extracted into a shared helper to remove duplication; implement a function (e.g., applyFolderHierarchy) that accepts organizeService, folderHierarchy, and optional explorerIconService, calls createFolderStructure, iterates docLinks to create links via organizeService.folderTree.folderNode$(folderId).value and folder.createLink('doc', ...), returns the root folderId or undefined, and catches/logs errors using logger.warn; then replace the repeated blocks in markdownZip, notion, and bear with a single call to this helper (guarding on folderHierarchy and organizeService) to obtain rootFolderId.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@blocksuite/affine/widgets/linked-doc/src/transformers/markdown.ts`:
- Around line 580-588: The allSameRoot check currently recomputes startsWith
over entries for each entry (causing O(n²)); move this work outside the
per-entry logic by computing the common root once: derive the candidate root
from folderParts[0] (or from the first entry) and evaluate a single allSameRoot
boolean by scanning entries once (e.g., entries.every(e =>
e.fullPath.startsWith(candidateRoot + '/'))), then if true slice folderParts =
folderParts.slice(1); update the code locations where allSameRoot, folderParts
and entries are referenced (the variables named folderParts, entries and the
allSameRoot check) to use this precomputed value so the loop becomes O(n).
In `@packages/frontend/core/src/desktop/dialogs/import/index.tsx`:
- Around line 636-661: The repeated folder-hierarchy creation logic should be
extracted into a shared helper to remove duplication; implement a function
(e.g., applyFolderHierarchy) that accepts organizeService, folderHierarchy, and
optional explorerIconService, calls createFolderStructure, iterates docLinks to
create links via organizeService.folderTree.folderNode$(folderId).value and
folder.createLink('doc', ...), returns the root folderId or undefined, and
catches/logs errors using logger.warn; then replace the repeated blocks in
markdownZip, notion, and bear with a single call to this helper (guarding on
folderHierarchy and organizeService) to obtain rootFolderId.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9291cab3-67fe-44b4-a1ff-d2565482f23d
⛔ Files ignored due to path filters (1)
yarn.lockis excluded by!**/yarn.lock,!**/*.lock
📒 Files selected for processing (10)
blocksuite/affine/inlines/preset/src/adapters/html/html-inline.tsblocksuite/affine/widgets/linked-doc/package.jsonblocksuite/affine/widgets/linked-doc/src/transformers/bear.tsblocksuite/affine/widgets/linked-doc/src/transformers/index.tsblocksuite/affine/widgets/linked-doc/src/transformers/markdown.tsblocksuite/playground/apps/_common/components/starter-debug-menu.tspackages/frontend/core/src/desktop/dialogs/import/index.tsxpackages/frontend/i18n/src/i18n-completenesses.jsonpackages/frontend/i18n/src/i18n.gen.tspackages/frontend/i18n/src/resources/en.json
✅ Files skipped from review due to trivial changes (5)
- packages/frontend/i18n/src/i18n-completenesses.json
- blocksuite/affine/widgets/linked-doc/package.json
- blocksuite/affine/widgets/linked-doc/src/transformers/index.ts
- blocksuite/affine/inlines/preset/src/adapters/html/html-inline.ts
- packages/frontend/i18n/src/resources/en.json
🚧 Files skipped from review as they are similar to previous changes (2)
- blocksuite/playground/apps/_common/components/starter-debug-menu.ts
- blocksuite/affine/widgets/linked-doc/src/transformers/bear.ts
Add support for importing Bear .bear2bk backup files, which are zipped TextBundle archives. The importer handles Bear-specific markdown extensions (highlights, callouts, tags), extracts metadata from info.json (creation/modification dates), and builds a folder hierarchy from nested tags. Also enhances the markdown zip importer to preserve folder structure from zip paths, and adds colored highlight support to the HTML mark adapter. Relates to toeverything#14115, toeverything#10003, toeverything#11286
Prevent markup injection in convertHighlights by escaping special characters before interpolating into <mark> elements. Add JSDoc docstrings to all functions in the Bear transformer module.
…dle errors convertGfmCallouts now processes line-by-line with code fence tracking, preventing callout markers inside fenced code blocks from being rewritten. The per-bundle import loop is wrapped in try/catch so a single failing note does not abort the entire backup import after earlier notes have already been written to the workspace.
…Bear import
Use Unicode property escapes (\p{L}\p{N}) for open-tag regex so tags
like #café, #日本, #täg/sub parse correctly. Lowercase file extensions
before extMimeMap lookup so IMG.PNG/photo.JPG resolve to the right MIME.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
Previously Bear tags were lowercased during deduplication and again during root-segment consolidation, so a tag like "WorkProject" would appear as "workproject" in AFFiNE. Keep the original capitalization of the first occurrence of each tag while still deduping case-insensitively.
Summary
.bear2bkbackup importer (TextBundle-based zip format)<mark data-color="...">) support to HTML adapterBear Import Details
Bear backups are zip archives of TextBundle directories. The importer:
==text==, callouts> [!NOTE], inline tags#tag)info.jsonmetadata#work/projects/alpha)Markdown Zip Folder Hierarchy
importMarkdownZipnow returns{ docIds, folderHierarchy }instead of justdocIds[], enabling the UI to recreate the zip's directory structure as AFFiNE folders.Related Issues
Test Plan
.bear2bkbackup file via the import dialogSummary by CodeRabbit
New Features
Bug Fixes / Improvements
Chores
Documentation