Fix exploring remote webknossos datasource-properties.jsons with paths

[fm3]

If the mag has a relative path, it must be resolved in the dataset root path.

Based on and thus blocked by #8844

URL of deployed dev instance (used for testing):

• https://fixexplorepaths.webknossos.xyz

Steps to test:

• Visit the served zarr-streaming datasource-properties.json at http://localhost:9000/data/zarr/<datasetId>/datasource-properties.json?token=secretSampleUserToken
• Should list relative paths
• Explore the dataset in the same webknossos http://localhost:9000/data/zarr/<datasetId>/ and add it (self-stream)
• Should be explorable and afterwards show data.
• Same for annotation zarr links (note: explore the whole annotation, not just one layer)

Issues:

• fixes Exploring datasets with explicit relative mag paths is broken #8811

---

• Added changelog entry (create a $PR_NUMBER.md file in unreleased_changes or use ./tools/create-changelog-entry.py)
• Considered common edge cases
• Needs datastore update after deployment

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

unreleased_changes/8897.md (1)

1-2: Clarify changelog entry and link the fixed issue

Spell out the exact behavior (relative mag path resolution target), scope (Zarr + annotations), and reference the issue.

Please confirm whether paths are resolved relative to the dataset root or to the datasource-properties.json URL; the PR text vs. issue #8811 differ.

 ### Fixed
-- Fixed a bug where exploring remote datasets served by WEBKNOSSOS would yield invalid paths.
+- Exploring remote WEBKNOSSOS Zarr datasets with relative mag paths now works: mag paths in `datasource-properties.json` are resolved relative to the `datasource-properties.json` URL (affects image and annotation layers). Fixes #8811.
+- When serving `datasource-properties.json` for WEBKNOSSOS-based Zarr streaming, emitted mag paths are pre-resolved so they remain usable regardless of the underlying storage location.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0ace83 and a8910d3.

📒 Files selected for processing (3)

• unreleased_changes/8897.md (1 hunks)
• webknossos-datastore/app/com/scalableminds/webknossos/datastore/explore/WebknossosZarrExplorer.scala (2 hunks)
• webknossos-tracingstore/app/com/scalableminds/webknossos/tracingstore/controllers/VolumeTracingZarrStreamingController.scala (3 hunks)

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-05-12T13:07:29.637Z

Learnt from: frcroth
PR: scalableminds/webknossos#8609
File: app/models/dataset/Dataset.scala:753-775
Timestamp: 2025-05-12T13:07:29.637Z
Learning: In the `updateMags` method of DatasetMagsDAO (Scala), the code handles different dataset types distinctly:
1. Non-WKW datasets have `magsOpt` populated and use the first branch which includes axisOrder, channelIndex, and credentialId.
2. WKW datasets will have `wkwResolutionsOpt` populated and use the second branch which includes cubeLength.
3. The final branch is a fallback for legacy data.
This ensures appropriate fields are populated for each dataset type.

Applied to files:

• webknossos-datastore/app/com/scalableminds/webknossos/datastore/explore/WebknossosZarrExplorer.scala

🧬 Code graph analysis (2)

webknossos-tracingstore/app/com/scalableminds/webknossos/tracingstore/controllers/VolumeTracingZarrStreamingController.scala (3)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/helpers/UPath.scala (2)

• UPath (54-96)
• fromLocalPath (80-80)

util/src/main/scala/com/scalableminds/util/geometry/Vec3Int.scala (1)

• toMagLiteral (40-43)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/dataformats/MagLocator.scala (2)

• MagLocator (10-15)
• MagLocator (17-19)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/explore/WebknossosZarrExplorer.scala (6)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/dataformats/MagLocator.scala (2)

• MagLocator (10-15)
• MagLocator (17-19)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/zarr3/Zarr3ArrayHeader.scala (3)

• Zarr3ArrayHeader (24-100)
• Zarr3ArrayHeader (162-310)
• FILENAME_ZARR_JSON (164-164)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/datavault/VaultPath.scala (2)

• VaultPath (19-97)
• toUPath (75-75)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/helpers/UPath.scala (2)

• UPath (54-96)
• resolvedIn (33-38)

webknossos-datastore/app/com/scalableminds/webknossos/datastore/models/datasource/DataFormat.scala (1)

• DataFormat (5-7)

util/src/main/scala/com/scalableminds/util/geometry/Vec3Int.scala (1)

• toMagLiteral (40-43)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: backend-tests
• GitHub Check: build-smoketest-push
• GitHub Check: frontend-tests

🔇 Additional comments (8)

webknossos-tracingstore/app/com/scalableminds/webknossos/tracingstore/controllers/VolumeTracingZarrStreamingController.scala (3)

30-30: LGTM! Added required imports for path construction.

The imports for UPath and Path are correctly added to support the new path construction logic in the zarrSource method.

Also applies to: 52-52

---

278-278: LGTM! Layer name derivation logic is sound.

The fallback from tracingName to tracingId provides a reasonable default when no explicit tracing name is provided.

---

285-292: LGTM! MagLocator path construction aligns with PR objectives.

The path construction using UPath.fromLocalPath(Path.of(s"./$layerName/${m.toMagLiteral(allowScalar = true)}")) correctly creates relative paths that will be resolved against the dataset root, addressing the core issue described in #8811.

The explicit specification of AxisOrder.cxyz ensures consistent data ordering for zarr streaming.

webknossos-datastore/app/com/scalableminds/webknossos/datastore/explore/WebknossosZarrExplorer.scala (5)

4-4: LGTM! Added necessary imports for enhanced path resolution.

The imports for Box types (Box, Failure, Full) and DataFormat support the new error handling and format-specific logic in the updated methods.

Also applies to: 11-11

---

26-28: LGTM! Updated method calls align with new signature requirements.

The calls to headerFilename(layer) and adaptMags with the new parameters (remotePath, layer.name, headerFilename, credentialId) correctly pass the dataset path and layer name needed for proper path resolution.

---

33-38: LGTM! Header filename determination supports both zarr formats.

The pattern matching on DataFormat correctly maps zarr and zarr3 formats to their respective header filenames, with appropriate error handling for unsupported formats.

---

40-48: LGTM! Enhanced adaptMags signature supports layered path construction.

The updated signature with remoteDatasetPath and layerName parameters enables proper path resolution relative to the dataset root, which is the core fix for the issue described in PR #8811.

---

50-63: LGTM! Path resolution logic correctly handles both existing and missing paths.

The implementation properly:

• Resolves existing paths relative to the remoteDatasetPath using path.resolvedIn(remoteDatasetPath.toUPath)
• Constructs new paths using the pattern remoteDatasetPath / layerName / magLiteral when no path exists
• Validates the constructed path by attempting to read the header file before returning it

This addresses the core issue where relative mag paths need to be resolved against the dataset root rather than imported verbatim.