Update documentation to reflect Zarr as the new default data format #8694
Conversation
…replacing WKW references
📝 WalkthroughWalkthroughThe documentation was updated throughout several files to reflect the adoption of Zarr3 as the default internal data format in WEBKNOSSOS, replacing WKW. Instructions, examples, and explanations were revised to indicate support for both Zarr and WKW formats, clarify conversion workflows, and update descriptions of supported data types. Changes
Possibly related PRs
Suggested reviewers
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (10)
docs/data/index.md (2)
13-13: Refine default format wording and punctuation.Use “Zarr v3” consistently and replace the hyphen with a colon (or semicolon) for clarity.
Example diff:
- The Zarr3 format is used for all internal voxel data representations - both for the raw (microscopy) image datasets and segmentations. + The Zarr v3 format is used for all internal voxel data representations: both raw microscopy image datasets and segmentations.
15-16: Improve upload conversion phrasing.Replace “given” with “provided” (or “if”), adjust punctuation, and clarify the flow.
Example diff:
- Any dataset uploaded to webknossos.org will automatically be converted to Zarr3 on upload - given its source file format is supported by WEBKNOSSOS. + Any dataset uploaded to webknossos.org will automatically be converted to Zarr v3 on upload, provided its source file format is supported by WEBKNOSSOS.docs/data/upload_ui.md (2)
14-14: Consistent naming and punctuation for supported formats.Add a comma and normalize format names (“Zarr” vs “Zarr v3”).
Example diff:
- Internally, WEBKNOSSOS uses the [Zarr3](./zarr.md) format by default to display your data. + Internally, WEBKNOSSOS uses the [Zarr v3](./zarr.md) format by default to display your data.
15-16: Clarify list of pre-supported formats.Use parallel structure and punctuation for readability.
Example diff:
- If your data is already in a data format like [WKW](./wkw.md), [Zarr or Zarr3](./zarr.md) you can simply drag your folder (or zip archive of that folder) into the upload view. + If your data is already in a supported format—[WKW](./wkw.md), [Zarr](./zarr.md), or [Zarr v3](./zarr.md)—you can simply drag your folder (or a zip archive) into the upload view.docs/data/image_stacks.md (3)
9-9: Specify default as Zarr v3.To avoid confusion, mention “Zarr v3” when referring to Zarr.
Example diff:
- Image stacks need to be converted to [Zarr](./zarr.md) or [WKW](./wkw.md) datasets for WEBKNOSSOS. + Image stacks need to be converted to [Zarr v3](./zarr.md) or [WKW](./wkw.md) datasets for WEBKNOSSOS.
63-63: Ensure CLI example aligns with default format.Update the comment to emphasize Zarr v3:
Example diff:
- After installing, you can convert image stacks to Zarr3 datasets with the following command: + After installing, you can convert image stacks to Zarr v3 datasets with the following command:
74-74: Fix article and comma in description.Add “a” before “directory” and insert a comma for readability.
Example diff:
- This snippet converts an image stack that is located in directory called `data/source` into a Zarr3 dataset which will be located at `data/target`. + This snippet converts an image stack located in a directory called `data/source` into a Zarr v3 dataset, which will be placed in `data/target`.docs/data/wkw.md (2)
4-4: Use “many” instead of “a lot of” for formality.Replace informal phrasing and improve flow.
Example diff:
- It works well for large datasets and is built with modern file systems in mind and drives a lot of WEBKNOSSOS datasets. + It works well for large datasets, is built with modern file systems in mind, and drives many WEBKNOSSOS datasets.
9-26: Correct Markdown table formatting and terminology.The current table has extra empty rows and misaligned columns. Also replace “til” with “till” or “until.”
Example outline for revised table:
| dtype | Color Layers | Segmentation Layers | |-----------------|-------------:|----------------------------------------------------| | uint8 | ✓ | ✓ | | uint16 | ✓ | ✓ | | uint24 (RGB) | ✓ | N/A | | uint32 | ✓ | ✓ | | uint64 | ✗ | ✓ (till 2⁵³−1) [#6921](https://github.com/scalableminds/webknossos/issues/6921) | | int8 | ✓ | ✓ | | int16 | ✓ | ✓ | | int32 | ✓ | ✓ | | int64 | ✗ | ✓ | | float32 | ✓ | ✗ | | float64 | ✗ | ✗ |docs/data/zarr.md (1)
5-5: Use present perfect for replacement statement.Adjust tense and improve readability.
Example diff:
- Therefore, Zarr is the new default data format in WEBKNOSSOS and replaced the previous [WKW](./wkw.md) format. + Therefore, Zarr has become the default data format in WEBKNOSSOS, replacing the previous [WKW](./wkw.md) format.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
docs/data/image_stacks.md(3 hunks)docs/data/index.md(1 hunks)docs/data/upload_ui.md(1 hunks)docs/data/wkw.md(1 hunks)docs/data/zarr.md(2 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/data/image_stacks.md
[uncategorized] ~74-~74: You might be missing the article “a” here.
Context: ...verts an image stack that is located in directory called data/source into a Zarr3 datas...
(AI_EN_LECTOR_MISSING_DETERMINER_A)
[uncategorized] ~74-~74: Possible missing comma found.
Context: ...ctory called data/source into a Zarr3 dataset which will be located at data/target....
(AI_HYDRA_LEO_MISSING_COMMA)
docs/data/wkw.md
[style] ~17-~17: The word ‘til’ with one ‘l’ is informal. Consider replacing it.
Context: ...| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalablemind...
(TIL)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: build-smoketest-push
- GitHub Check: frontend-tests
- GitHub Check: backend-tests
🔇 Additional comments (2)
docs/data/zarr.md (2)
133-133: Approve note on v2 dataset output.The clarification about
--data-format zarrproducing unsharded Zarr v2 datasets is accurate.
139-139: Fix library name spelling.Correct “WEBKNOSSSO” to “WEBKNOSSOS” in the Python conversion intro.
Example diff:
- You can use the free [WEBKNOSSSO Python library](https://docs.webknossos.org/webknossos-py) to convert image stacks to Zarr3 or integrate the conversion as part of an existing workflow. + You can use the free [WEBKNOSSOS Python library](https://docs.webknossos.org/webknossos-py) to convert image stacks to Zarr v3 or integrate the conversion as part of an existing workflow.Likely an incorrect or invalid review comment.
docs/data/wkw.md
Outdated
| * Grayscale data (8 Bit, 16 Bit, Float), also referred to as `color` data | ||
| * RGB data (24 Bit) | ||
| * Segmentation data (8 Bit, 16 Bit, 32 Bit) | ||
| The following table shows an overview of supported `dtypes` per layer: |
There was a problem hiding this comment.
Der DType support is unabhängig von WKW oder? Das kann WK mit alle Quellen (Zarr, WKW, Neuroglancer etc) , oder? Vielleicht gibt's noch einen besseren Ort für die Tabelle
There was a problem hiding this comment.
Ich hätte jetzt erst einmal upload_ui.md vorgeschlagen, da stehen schon infos über unterstützte Bildformate, vielleicht kann man da auch gleich auf die unterstützten dtypes eingehen. Was meinst du dazu?
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (8)
docs/data/upload_ui.md (2)
14-17: Improve clarity and consistency of format mentions.The phrasing around Zarr v2 and Zarr3 is ambiguous and missing commas. Consider explicitly listing formats and updating “Zarr or Zarr3” to “Zarr v2 or Zarr3” with proper punctuation.
Example diff:
- Internally, WEBKNOSSOS uses the [Zarr3](./zarr.md) format by default to display your data. - If your data is already in a data format like [WKW](./wkw.md), [Zarr or Zarr3](./zarr.md) you can simply drag your folder (or zip archive of that folder) into the upload view. - If your data is not in WKW or Zarr format, you can either: + Internally, WEBKNOSSOS uses the [Zarr3](./zarr.md) format by default to display your data. + If your data is already in a supported format (e.g., [WKW](./wkw.md), [Zarr v2](./zarr.md), or [Zarr3](./zarr.md)), you can simply drag your folder (or a zip archive of that folder) into the upload view. + If your data is not in WKW, Zarr v2, or Zarr3 format, you can either:
37-38: Refine sentence for clarity and grammar.The sentence is wordy and missing a comma. Consider:
- We support a variety of data types for the uploaded data. To make sure that your data can be uploaded to WEBKNOSSOS take a look into this table of supported data types for color and segmentation layers: + We support a variety of data types for uploaded datasets. To ensure your data can be uploaded to WEBKNOSSOS, refer to this table of supported data types for color and segmentation layers:docs/data/image_stacks.md (3)
9-9: Clarify default conversion format.The text refers to generic “Zarr” but elsewhere uses “Zarr3” as the default. To avoid confusion, update to:
- Image stacks need to be converted to [Zarr](./zarr.md) or [WKW](./wkw.md) datasets for WEBKNOSSOS. + Image stacks need to be converted to [Zarr3](./zarr.md) or [WKW](./wkw.md) datasets for WEBKNOSSOS.
63-63: Improve sentence grammar.Add the missing article and comma:
- After installing, you can convert image stacks to Zarr3 datasets with the following command: + After installing, you can convert an image stack in a directory called `data/source` into a Zarr3 dataset at `data/target` using the following command:
82-82: Unify Zarr version notation.This line uses “Zarr v3” but other docs use “Zarr3”. Choose one notation across all documentation for consistency.
docs/data/zarr.md (3)
5-5: Use present perfect tense and specify Zarr3 explicitly.Consider updating for grammatical accuracy and clarity:
- We strongly believe in this community-driven, cloud-native data format for n-dimensional datasets. Therefore, Zarr is the new default data format in WEBKNOSSOS and replaced the previous [WKW](./wkw.md) format. + We strongly believe in this community-driven, cloud-native format for n-dimensional datasets. Therefore, Zarr3 has replaced the previous [WKW](./wkw.md) format as WEBKNOSSOS’s default.
131-131: Fix plurality and spacing in chunk size.The chunk size should be plural and match comma spacing:
- …chunk size of (64,64,64) voxel. + …chunk size of (64, 64, 64) voxels.
139-139: Unify Zarr version notation.This line uses “Zarr3” while earlier text and CLI examples refer to “Zarr v3”. Standardize to one format across docs.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
docs/data/image_stacks.md(3 hunks)docs/data/upload_ui.md(2 hunks)docs/data/wkw.md(1 hunks)docs/data/zarr.md(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/data/wkw.md
🧰 Additional context used
🪛 LanguageTool
docs/data/image_stacks.md
[uncategorized] ~74-~74: You might be missing the article “a” here.
Context: ...verts an image stack that is located in directory called data/source into a Zarr3 datas...
(AI_EN_LECTOR_MISSING_DETERMINER_A)
[uncategorized] ~74-~74: Possible missing comma found.
Context: ...ctory called data/source into a Zarr3 dataset which will be located at data/target....
(AI_HYDRA_LEO_MISSING_COMMA)
docs/data/upload_ui.md
[style] ~37-~37: Consider using a more concise synonym.
Context: ...d) - N5 datasets We support a variety of data types for the uploaded data. To ma...
(A_VARIETY_OF)
[uncategorized] ~37-~37: Possible missing comma found.
Context: ... sure that your data can be uploaded to WEBKNOSSOS take a look into this table of supporte...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~37-~37: The preposition “at” seems more likely in this position.
Context: ...n be uploaded to WEBKNOSSOS take a look into this table of supported data types for ...
(AI_EN_LECTOR_REPLACEMENT_PREPOSITION)
[style] ~45-~45: The word ‘til’ with one ‘l’ is informal. Consider replacing it.
Context: ...| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalablemind...
(TIL)
🪛 markdownlint-cli2 (0.17.2)
docs/data/upload_ui.md
46-46: Table column count
Expected: 3; Actual: 4; Too many cells, extra data will be missing
(MD056, table-column-count)
51-51: Table column count
Expected: 3; Actual: 4; Too many cells, extra data will be missing
(MD056, table-column-count)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: build-smoketest-push
- GitHub Check: backend-tests
- GitHub Check: frontend-tests
| | dtype | Color Layers | Segmentation Layers | | ||
| |------------|------------|------------| | ||
| | uint8 | ✓ | ✓ | | ||
| | uint16 | ✓ | ✓ | | ||
| | uint24 rgb | ✓ | does not apply | | ||
| | uint32 | ✓ | ✓ | | ||
| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalableminds/webknossos/issues/6921) | | ||
| | | | | | | ||
| | int8 | ✓ | ✓ | | ||
| | int16 | ✓ | ✓ | | ||
| | int32 | ✓ | ✓ | | ||
| | int64 | ✗ | ✓ | | ||
| | | | | | | ||
| | float | ✓ | ✗ | | ||
| | double | ✗ | ✗ | | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Fix markdown table column count and dtype naming.
The blank rows at lines 46 and 51 introduce extra columns, violating MD056. Remove those empty rows. Also consider renaming “uint24 rgb” to “uint24 (RGB)” for consistency.
Example diff:
-| | | | |
-| | | | |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | dtype | Color Layers | Segmentation Layers | | |
| |------------|------------|------------| | |
| | uint8 | ✓ | ✓ | | |
| | uint16 | ✓ | ✓ | | |
| | uint24 rgb | ✓ | does not apply | | |
| | uint32 | ✓ | ✓ | | |
| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalableminds/webknossos/issues/6921) | | |
| | | | | | | |
| | int8 | ✓ | ✓ | | |
| | int16 | ✓ | ✓ | | |
| | int32 | ✓ | ✓ | | |
| | int64 | ✗ | ✓ | | |
| | | | | | | |
| | float | ✓ | ✗ | | |
| | double | ✗ | ✗ | | |
| | dtype | Color Layers | Segmentation Layers | | |
| |------------|------------|------------| | |
| | uint8 | ✓ | ✓ | | |
| | uint16 | ✓ | ✓ | | |
| | uint24 rgb | ✓ | does not apply | | |
| | uint32 | ✓ | ✓ | | |
| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalableminds/webknossos/issues/6921) | | |
| | int8 | ✓ | ✓ | | |
| | int16 | ✓ | ✓ | | |
| | int32 | ✓ | ✓ | | |
| | int64 | ✗ | ✓ | | |
| | float | ✓ | ✗ | | |
| | double | ✗ | ✗ | |
🧰 Tools
🪛 LanguageTool
[style] ~45-~45: The word ‘til’ with one ‘l’ is informal. Consider replacing it.
Context: ...| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalablemind...
(TIL)
🪛 markdownlint-cli2 (0.17.2)
46-46: Table column count
Expected: 3; Actual: 4; Too many cells, extra data will be missing
(MD056, table-column-count)
51-51: Table column count
Expected: 3; Actual: 4; Too many cells, extra data will be missing
(MD056, table-column-count)
🤖 Prompt for AI Agents
In docs/data/upload_ui.md around lines 39 to 54, the markdown table has blank
rows at lines 46 and 51 that create extra columns and violate markdown syntax.
Remove these empty rows to fix the column count. Also, rename the dtype "uint24
rgb" to "uint24 (RGB)" for consistent naming style.
There was a problem hiding this comment.
@markbader I believe CodeRabbit might have a point here. The empty rows have 5x | instead of the 4x separators for regular rows.
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (3)
docs/data/upload_ui.md (3)
43-45: Rename and reformat the uint24 RGB entry.The dtype label "uint24 rgb" is inconsistent; rename to "uint24 (RGB)" for clarity and consistency with other entries.
46-46: Remove the blank row.The empty table row at line 46 introduces an extra column and breaks markdown table syntax.
51-51: Remove the blank row.The empty table row at line 51 introduces an extra column and violates markdown table syntax.
🧹 Nitpick comments (1)
docs/data/upload_ui.md (1)
37-37: Refine phrasing for clarity and flow.Consider rewriting to:
"To ensure your data can be uploaded to WEBKNOSSOS, review this table of supported data types for color and segmentation layers:"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/data/upload_ui.md(2 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/data/upload_ui.md
[style] ~37-~37: Consider using a more concise synonym.
Context: ...d) - N5 datasets We support a variety of data types for the uploaded data. To ma...
(A_VARIETY_OF)
[uncategorized] ~37-~37: Possible missing comma found.
Context: ... sure that your data can be uploaded to WEBKNOSSOS take a look into this table of supporte...
(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~37-~37: The preposition “at” seems more likely in this position.
Context: ...n be uploaded to WEBKNOSSOS take a look into this table of supported data types for ...
(AI_EN_LECTOR_REPLACEMENT_PREPOSITION)
[style] ~45-~45: The word ‘til’ with one ‘l’ is informal. Consider replacing it.
Context: ...| | uint64 | ✗ | (✓) [(til 2⁵³−1)](https://github.com/scalablemind...
(TIL)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: build-smoketest-push
- GitHub Check: backend-tests
🔇 Additional comments (6)
docs/data/upload_ui.md (6)
14-15: Updated default format docs look good.Switching the default display format to Zarr3 and including references to WKW and Zarr is clear and accurate.
17-17: Guidance for unsupported formats is clear.The "not in WKW or Zarr format" phrasing correctly directs users toward conversion options.
39-40: Table header is clear and well-structured.No issues with column names or alignment.
41-42: Data type rows for uint8 and uint16 are correct.Entries accurately reflect support for color and segmentation layers.
47-50: Middle table entries are correctly formatted.Entries for int8 through int32 accurately reflect support status.
52-54: Floating-point type entries look good.Support indications for float and double are accurate.
2 participants
This PR updates some documentation pages, to replace WKW with the new Zarr3 default data format.