Merge branch 'users-can-change-email' of github.com:scalableminds/webknossos into users-can-change-email

Author: MichaelBuessemeyer

CHANGELOG.released.md

@@ -7,6 +7,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 For upgrade instructions, please check the [migration guide](MIGRATIONS.released.md).
 
+## [25.06.2](https://github.com/scalableminds/webknossos/releases/tag/25.06.2) - 2025-06-17
+[Commits](https://github.com/scalableminds/webknossos/compare/25.06.1...25.06.2)
+
+### Fixed
+- Fixed a regression that led to incorrect behavior when trying to jump to the last branchpoint even though no branchpoint existed. [#8695](https://github.com/scalableminds/webknossos/pull/8695)
+
 ## [25.06.1](https://github.com/scalableminds/webknossos/releases/tag/25.06.1) - 2025-06-03
 [Commits](https://github.com/scalableminds/webknossos/compare/25.06.0...25.06.1)
 

CHANGELOG.unreleased.md

@@ -8,17 +8,23 @@ and this project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MIC
 For upgrade instructions, please check the [migration guide](MIGRATIONS.released.md).
 
 ## Unreleased
-[Commits](https://github.com/scalableminds/webknossos/compare/25.06.1...HEAD)
+[Commits](https://github.com/scalableminds/webknossos/compare/25.06.2...HEAD)
 
 ### Added
+- In shared annotations with multiple authors, some changes are now stored per user. This means that other users won’t see all those changes if their own diverge. This includes the current position and zoom, visibilities of trees, bounding boxes, and segments (as specified with the checkboxes in the lists), as well as which groups are expanded in the lists. The annotation owner’s user state is used as a fallback for users who haven’t explicitly changed these values themselves. [#8542](https://github.com/scalableminds/webknossos/pull/8542)
 - Added the ability to duplicate trees in skeleton annotations. Users can create a copy of any tree (including all nodes, edges, and properties) via the context menu in the skeleton tab. [#8662](https://github.com/scalableminds/webknossos/pull/8662)
 - Meshes are now reloaded using their previous opacity value. [#8622](https://github.com/scalableminds/webknossos/pull/8622)
 
 ### Changed
+- The datasource-properties.json as exposed in zarr streaming routes and volume annotation download now includes an explicit “path” attribute for every mag. The supplied path is relative to the location of the datasource-properties.json itself, e.g. `./color/2-2-1`. [#8518](https://github.com/scalableminds/webknossos/pull/8518)
 
 ### Fixed
 - Improved efficiency of saving bounding box related changes. [#8492](https://github.com/scalableminds/webknossos/pull/8492)
 - When deleting a dataset, its caches are cleared, so that if a new dataset by the same name is uploaded afterwards, only new data is loaded. [#8638](https://github.com/scalableminds/webknossos/pull/8638)
+- Fixed the contrast of the WelcomeToast buttons. Updated `antd` to version `5.22`.[#8688](https://github.com/scalableminds/webknossos/pull/8688)
+- Fixed a race condition when starting proofreading with a split action. [#8676](https://github.com/scalableminds/webknossos/pull/8676)
+- Fixed that activating a mapping got stuck when a dataset was opened in "view" mode. [#8687](https://github.com/scalableminds/webknossos/pull/8687)
+- Fixed a regression that led to incorrect behavior when trying to jump to the last branchpoint even though no branchpoint existed. [#8695](https://github.com/scalableminds/webknossos/pull/8695)
 
 ### Removed
 

MIGRATIONS.released.md

@@ -6,6 +6,12 @@ See `MIGRATIONS.unreleased.md` for the changes which are not yet part of an offi
 This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 
+## [25.06.2](https://github.com/scalableminds/webknossos/releases/tag/25.06.2) - 2025-06-17
+[Commits](https://github.com/scalableminds/webknossos/compare/25.06.1...25.06.2)
+
+### Postgres Evolutions:
+None.
+
 ## [25.06.1](https://github.com/scalableminds/webknossos/releases/tag/25.06.1) - 2025-06-03
 [Commits](https://github.com/scalableminds/webknossos/compare/25.06.0...25.06.1)
 

MIGRATIONS.unreleased.md

@@ -6,7 +6,10 @@ This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 
 ## Unreleased
-[Commits](https://github.com/scalableminds/webknossos/compare/25.06.1...HEAD)
+[Commits](https://github.com/scalableminds/webknossos/compare/25.06.2...HEAD)
+
+### Postgres Evolutions:
+- The default thread pool size was increased from 5 to 10 times the number of available CPUs (capped at 1000). Note that wk may need slightly more memory because of this. [#8686](https://github.com/scalableminds/webknossos/pull/8686)
 
 ### Postgres Evolutions:
 - [134-dataset-layer-attachments.sql](conf/evolutions/134-dataset-layer-attachments.sql)

app/controllers/AnnotationController.scala

@@ -420,6 +420,8 @@ class AnnotationController @Inject()(
       newAnnotationProto <- tracingStoreClient.duplicateAnnotation(
         annotation._id,
         newAnnotationId,
+        annotation._user,
+        user._id,
         version = None,
         isFromTask = annotation._task.isDefined,
         datasetBoundingBox = dataSource.map(_.boundingBox)

app/controllers/AnnotationIOController.scala

@@ -413,7 +413,7 @@ class AnnotationIOController @Inject()(
 
   private def downloadExplorational(annotationId: ObjectId,
                                     typ: String,
-                                    issuingUser: Option[User],
+                                    requestingUser: Option[User],
                                     version: Option[Long],
                                     skipVolumeData: Boolean,
                                     volumeDataZipFormat: VolumeDataZipFormat)(implicit ctx: DBAccessContext) = {
@@ -425,10 +425,12 @@ class AnnotationIOController @Inject()(
         tracingStoreClient <- tracingStoreService.clientFor(dataset)
         fetchedAnnotationLayers <- Fox.serialCombined(annotation.skeletonAnnotationLayers)(
           tracingStoreClient.getSkeletonTracing(annotation._id, _, version))
-        user <- userService.findOneCached(annotation._user)(GlobalAccessContext)
+        annotationProto <- tracingStoreClient.getAnnotationProto(annotation._id, version)
+        annotationOwner <- userService.findOneCached(annotation._user)(GlobalAccessContext)
         taskOpt <- Fox.runOptional(annotation._task)(taskDAO.findOne)
         nmlStream = nmlWriter.toNmlStream(
           "temp",
+          annotationProto,
           fetchedAnnotationLayers,
           Some(annotation),
           dataset.voxelSize,
@@ -437,10 +439,11 @@ class AnnotationIOController @Inject()(
           conf.Http.uri,
           dataset.name,
           dataset._id,
-          Some(user),
+          annotationOwner,
           taskOpt,
           skipVolumeData,
-          volumeDataZipFormat
+          volumeDataZipFormat,
+          requestingUser
         )
         nmlTemporaryFile = tempFileService.create()
         temporaryFileStream = new BufferedOutputStream(new FileOutputStream(new File(nmlTemporaryFile.toString)))
@@ -467,10 +470,12 @@ class AnnotationIOController @Inject()(
           skeletonAnnotationLayer =>
             tracingStoreClient.getSkeletonTracing(annotation._id, skeletonAnnotationLayer, version)
         } ?~> "annotation.download.fetchSkeletonLayer.failed"
-        user <- userService.findOneCached(annotation._user)(GlobalAccessContext) ?~> "annotation.download.findUser.failed"
+        annotationOwner <- userService.findOneCached(annotation._user)(GlobalAccessContext) ?~> "annotation.download.findUser.failed"
         taskOpt <- Fox.runOptional(annotation._task)(taskDAO.findOne(_)(GlobalAccessContext)) ?~> "task.notFound"
+        annotationProto <- tracingStoreClient.getAnnotationProto(annotation._id, version)
         nmlStream = nmlWriter.toNmlStream(
           name,
+          annotationProto,
           fetchedSkeletonLayers ::: fetchedVolumeLayers,
           Some(annotation),
           dataset.voxelSize,
@@ -479,10 +484,11 @@ class AnnotationIOController @Inject()(
           conf.Http.uri,
           dataset.name,
           dataset._id,
-          Some(user),
+          annotationOwner,
           taskOpt,
           skipVolumeData,
-          volumeDataZipFormat
+          volumeDataZipFormat,
+          requestingUser
         )
         temporaryFile = tempFileService.create()
         zipper = ZipIO.startZip(new BufferedOutputStream(new FileOutputStream(new File(temporaryFile.toString))))
@@ -520,13 +526,13 @@ class AnnotationIOController @Inject()(
         zipMimeType
 
     for {
-      annotation <- provider.provideAnnotation(typ, annotationId, issuingUser) ~> NOT_FOUND
+      annotation <- provider.provideAnnotation(typ, annotationId, requestingUser) ~> NOT_FOUND
       restrictions <- provider.restrictionsFor(typ, annotationId) ?~> "annotation.restrictions.unavailable"
       name <- provider.nameFor(annotation) ?~> "annotation.name.impossible"
       fileExtension = exportExtensionForAnnotation(annotation)
       fileName = name + fileExtension
       mimeType = exportMimeTypeForAnnotation(annotation)
-      _ <- restrictions.allowDownload(issuingUser) ?~> "annotation.download.notAllowed" ~> FORBIDDEN
+      _ <- restrictions.allowDownload(requestingUser) ?~> "annotation.download.notAllowed" ~> FORBIDDEN
       dataset <- datasetDAO.findOne(annotation._dataset)(GlobalAccessContext) ?~> "dataset.notFoundForAnnotation" ~> NOT_FOUND
       organization <- organizationDAO.findOne(dataset._organization)(GlobalAccessContext) ?~> "organization.notFound" ~> NOT_FOUND
       temporaryFile <- annotationToTemporaryFile(dataset, annotation, name, organization._id) ?~> "annotation.writeTemporaryFile.failed"
@@ -538,13 +544,13 @@ class AnnotationIOController @Inject()(
     }
   }
 
-  private def downloadProject(projectId: ObjectId, userOpt: Option[User], skipVolumeData: Boolean)(
+  private def downloadProject(projectId: ObjectId, requestingUserOpt: Option[User], skipVolumeData: Boolean)(
       implicit ctx: DBAccessContext,
       m: MessagesProvider) =
     for {
-      user <- userOpt.toFox ?~> Messages("notAllowed") ~> FORBIDDEN
+      requestingUser <- requestingUserOpt.toFox ?~> Messages("notAllowed") ~> FORBIDDEN
       project <- projectDAO.findOne(projectId) ?~> Messages("project.notFound", projectId) ~> NOT_FOUND
-      _ <- Fox.assertTrue(userService.isTeamManagerOrAdminOf(user, project._team)) ?~> "notAllowed" ~> FORBIDDEN
+      _ <- Fox.assertTrue(userService.isTeamManagerOrAdminOf(requestingUser, project._team)) ?~> "notAllowed" ~> FORBIDDEN
       annotations <- annotationDAO.findAllFinishedForProject(projectId)
       zipTempFilePath <- annotationService.zipAnnotations(annotations,
                                                           project.name,
@@ -556,7 +562,7 @@ class AnnotationIOController @Inject()(
                   fileName = _ => Some(TextUtils.normalize(project.name + "_nmls.zip")))
     }
 
-  private def downloadTask(taskId: ObjectId, userOpt: Option[User], skipVolumeData: Boolean)(
+  private def downloadTask(taskId: ObjectId, requestingUserOpt: Option[User], skipVolumeData: Boolean)(
       implicit ctx: DBAccessContext,
       m: MessagesProvider) = {
     def createTaskZip(task: Task): Fox[Path] = annotationService.annotationsFor(task._id).flatMap { annotations =>
@@ -566,10 +572,10 @@ class AnnotationIOController @Inject()(
     }
 
     for {
-      user <- userOpt.toFox ?~> Messages("notAllowed") ~> FORBIDDEN
+      requestingUser <- requestingUserOpt.toFox ?~> Messages("notAllowed") ~> FORBIDDEN
       task <- taskDAO.findOne(taskId) ?~> Messages("task.notFound") ~> NOT_FOUND
       project <- projectDAO.findOne(task._project) ?~> Messages("project.notFound") ~> NOT_FOUND
-      _ <- Fox.assertTrue(userService.isTeamManagerOrAdminOf(user, project._team)) ?~> Messages("notAllowed") ~> FORBIDDEN
+      _ <- Fox.assertTrue(userService.isTeamManagerOrAdminOf(requestingUser, project._team)) ?~> Messages("notAllowed") ~> FORBIDDEN
       zipTempFilePath <- createTaskZip(task)
     } yield {
       Ok.sendPath(zipTempFilePath,
@@ -578,7 +584,7 @@ class AnnotationIOController @Inject()(
     }
   }
 
-  private def downloadTaskType(taskTypeId: ObjectId, userOpt: Option[User], skipVolumeData: Boolean)(
+  private def downloadTaskType(taskTypeId: ObjectId, requestingUserOpt: Option[User], skipVolumeData: Boolean)(
       implicit ctx: DBAccessContext,
       m: MessagesProvider) = {
     def createTaskTypeZip(taskType: TaskType) =
@@ -593,9 +599,9 @@ class AnnotationIOController @Inject()(
       } yield zip
 
     for {
-      user <- userOpt.toFox ?~> Messages("notAllowed") ~> FORBIDDEN
+      requestingUser <- requestingUserOpt.toFox ?~> Messages("notAllowed") ~> FORBIDDEN
       taskType <- taskTypeDAO.findOne(taskTypeId) ?~> "taskType.notFound" ~> NOT_FOUND
-      _ <- Fox.assertTrue(userService.isTeamManagerOrAdminOf(user, taskType._team)) ?~> "notAllowed" ~> FORBIDDEN
+      _ <- Fox.assertTrue(userService.isTeamManagerOrAdminOf(requestingUser, taskType._team)) ?~> "notAllowed" ~> FORBIDDEN
       zipTempFilePath <- createTaskTypeZip(taskType)
     } yield {
       Ok.sendPath(zipTempFilePath,

app/controllers/TaskController.scala

@@ -67,7 +67,8 @@ class TaskController @Inject()(taskCreationService: TaskCreationService,
         taskParametersFull <- taskCreationService.createTracingsFromBaseAnnotations(taskParametersWithIds,
                                                                                     taskType,
                                                                                     dataset,
-                                                                                    dataSource)
+                                                                                    dataSource,
+                                                                                    request.identity._id)
         skeletonBaseOpts: List[Option[SkeletonTracing]] = taskCreationService.createTaskSkeletonTracingBases(
           taskParametersFull)
         volumeBaseOpts: List[Option[(VolumeTracing, Option[File])]] <- taskCreationService.createTaskVolumeTracingBases(

app/models/annotation/AnnotationLayerPrecedence.scala

@@ -2,14 +2,16 @@ package models.annotation
 
 import com.scalableminds.util.objectid.ObjectId
 import com.scalableminds.util.tools.{Fox, FoxImplicits}
-import com.scalableminds.webknossos.datastore.SkeletonTracing.SkeletonTracing
-import com.scalableminds.webknossos.datastore.VolumeTracing.VolumeTracing
+import com.scalableminds.webknossos.datastore.IdWithBool.Id32WithBool
+import com.scalableminds.webknossos.datastore.SkeletonTracing.{SkeletonTracing, SkeletonUserStateProto}
+import com.scalableminds.webknossos.datastore.VolumeTracing.{VolumeTracing, VolumeUserStateProto}
 import com.scalableminds.webknossos.datastore.geometry.{
   AdditionalCoordinateProto,
   NamedBoundingBoxProto,
   Vec3DoubleProto,
   Vec3IntProto
 }
+import com.scalableminds.webknossos.datastore.helpers.SkeletonTracingDefaults
 import com.scalableminds.webknossos.datastore.models.annotation.{
   AnnotationLayer,
   AnnotationLayerType,
@@ -27,6 +29,7 @@ case class RedundantTracingProperties(
     zoomLevel: Double,
     userBoundingBoxes: Seq[NamedBoundingBoxProto],
     editPositionAdditionalCoordinates: Seq[AdditionalCoordinateProto],
+    userStateBoundingBoxVisibilities: Map[String, Seq[Id32WithBool]] // UserId → Seq(bboxId, bboxIsVisible)
 )
 
 trait AnnotationLayerPrecedence extends FoxImplicits {
@@ -54,7 +57,8 @@ trait AnnotationLayerPrecedence extends FoxImplicits {
         editRotation = p.editRotation,
         zoomLevel = p.zoomLevel,
         userBoundingBoxes = p.userBoundingBoxes,
-        editPositionAdditionalCoordinates = p.editPositionAdditionalCoordinates
+        editPositionAdditionalCoordinates = p.editPositionAdditionalCoordinates,
+        userStates = adaptSkeletonUserStates(skeletonTracing.userStates, p)
       )
     }.getOrElse(skeletonTracing)
 
@@ -66,10 +70,61 @@ trait AnnotationLayerPrecedence extends FoxImplicits {
         editRotation = p.editRotation,
         zoomLevel = p.zoomLevel,
         userBoundingBoxes = p.userBoundingBoxes,
-        editPositionAdditionalCoordinates = p.editPositionAdditionalCoordinates
+        editPositionAdditionalCoordinates = p.editPositionAdditionalCoordinates,
+        userStates = adaptVolumeUserStates(volumeTracing.userStates, p)
       )
     }.getOrElse(volumeTracing)
 
+  private def adaptSkeletonUserStates(
+      userStates: Seq[SkeletonUserStateProto],
+      oldPrecedenceLayerProperties: RedundantTracingProperties): Seq[SkeletonUserStateProto] = {
+    val adaptedExistingUserStates = userStates.map { userState =>
+      val userId = userState.userId
+      oldPrecedenceLayerProperties.userStateBoundingBoxVisibilities.get(userId) match {
+        case None => userState
+        case Some(precedenceBboxVisibilities) =>
+          userState.copy(boundingBoxVisibilities = precedenceBboxVisibilities)
+      }
+    }
+    // We also have to create new user states for the users the old precedence layer has, but the new precedence layer is missing.
+    val newUserPrecedenceProperties = oldPrecedenceLayerProperties.userStateBoundingBoxVisibilities.filter(tuple =>
+      !userStates.exists(_.userId == tuple._1))
+    val newUserStates = newUserPrecedenceProperties.map {
+      case (userId: String, boundingBoxVisibilities: Seq[Id32WithBool]) =>
+        SkeletonTracingDefaults
+          .emptyUserState(userId)
+          .copy(
+            boundingBoxVisibilities = boundingBoxVisibilities
+          )
+    }
+    adaptedExistingUserStates ++ newUserStates
+  }
+
+  private def adaptVolumeUserStates(
+      userStates: Seq[VolumeUserStateProto],
+      oldPrecedenceLayerProperties: RedundantTracingProperties): Seq[VolumeUserStateProto] = {
+    val adaptedExistingUserStates = userStates.map { userState =>
+      val userId = userState.userId
+      oldPrecedenceLayerProperties.userStateBoundingBoxVisibilities.get(userId) match {
+        case None => userState
+        case Some(precedenceBboxVisibilities) =>
+          userState.copy(boundingBoxVisibilities = precedenceBboxVisibilities)
+      }
+    }
+    // We also have to create new user states for the users the old precedence layer has, but the new precedence layer is missing.
+    val newUserPrecedenceProperties = oldPrecedenceLayerProperties.userStateBoundingBoxVisibilities.filter(tuple =>
+      !userStates.exists(_.userId == tuple._1))
+    val newUserStates = newUserPrecedenceProperties.map {
+      case (userId: String, boundingBoxVisibilities: Seq[Id32WithBool]) =>
+        VolumeTracingDefaults
+          .emptyUserState(userId)
+          .copy(
+            boundingBoxVisibilities = boundingBoxVisibilities
+          )
+    }
+    adaptedExistingUserStates ++ newUserStates
+  }
+
   protected def getOldPrecedenceLayerProperties(existingAnnotationId: Option[ObjectId],
                                                 existingAnnotationLayers: List[AnnotationLayer],
                                                 previousVersion: Option[Long],
@@ -138,7 +193,8 @@ trait AnnotationLayerPrecedence extends FoxImplicits {
           s.zoomLevel,
           s.userBoundingBoxes ++ s.userBoundingBox.map(
             com.scalableminds.webknossos.datastore.geometry.NamedBoundingBoxProto(0, None, None, None, _)),
-          s.editPositionAdditionalCoordinates
+          s.editPositionAdditionalCoordinates,
+          s.userStates.map(userState => (userState.userId, userState.boundingBoxVisibilities)).toMap
         )
       case Right(v) =>
         RedundantTracingProperties(
@@ -147,7 +203,8 @@ trait AnnotationLayerPrecedence extends FoxImplicits {
           v.zoomLevel,
           v.userBoundingBoxes ++ v.userBoundingBox.map(
             com.scalableminds.webknossos.datastore.geometry.NamedBoundingBoxProto(0, None, None, None, _)),
-          v.editPositionAdditionalCoordinates
+          v.editPositionAdditionalCoordinates,
+          v.userStates.map(userState => (userState.userId, userState.boundingBoxVisibilities)).toMap
         )
     }
 }