The implementation of CytoBrowser can be divided into two main parts: the server side and the client side. The server implementation consists of the cytobrowser.js file in the root directory as well as the files in the server/ directory. The client files can be found in public/, and the client-side JavaScript code makes heavy use of the reveal module pattern for encapsulation. This document describes a few key aspects of the system, what modules they use, and how they are connected.
The following section describes how the server is started, and what happens when a client connects to it.
Before the CytoBrowser client can be opened, the server has to be started. The server is run with Node.js, which will have to be installed first. The server also uses a couple of Node modules, which can be installed with npm by running the command npm install. With the modules installed, the server can be started with the command node cytobrowser.js [address] [port], which specifies which interface the server should listen on. When starting the server, three more command line arguments, -d, -c and -m, can be provided to specify directories for the image data, collaboration storage data and metadata. For example, running node cytobrowser.js -d ./data -c ./collab_storage -m ./metadata/json 127.0.0.1 8080 starts a server on 127.0.0.1:8080 with image data in ./data, automatically saved collaboration sessions in ./collab_storage, and metadata in ./metadata/json.
A couple of things happen when the server is started. It sets up endpoints for client requests, including WebSocket endpoints for collaboration. It then sets up a static path for ./public. It also initializes the different modules that will be used.
The client can be opened once the server has been started. As ./public is being statically served by the server, going to the address specified when starting the server will open up ./public/index.html. This page loads all the necessary client-side javascript, and then runs a short script once it has been loaded. The script calls the initUI() function in .public/js/tmappUI.js to set up the parts of the user interface that need to be added programmatically, and also calls the init() function in ./public/js/tmapp.js in order to initialize the OpenSeadragon instance.
Before calling tmapp.init(), the script in ./public/index.html looks at the search parameters specified in the URL. These are sent as arguments to tmapp.init() in order to specify the initial state of the viewer, the collaboration being joined, and the image being opened.
The tmapp.init() begins by making a GET request to the api/images endpoint on the server. This endpoint provides a JSON response that describes all the images available in the data directory specified when starting the server. The information described by these images are the image names, the different z levels available for them, and the paths to two images that can be used as thumbnails when previewing the image. The server finds this information programmatically using the module availableImages.js. Once this information has been retrieved by the client, the image information is stored in tmapp, tmappUI.updateImages() is called to set up the images in the image browser, and any actions necessitated by the URL search parameters, i.e. opening an image or joining a collaboration, are carried out.
The client-side tmapp module contains the function openImage(), which should be used whenever the image has to be changed. This function uses data retrieved in tmapp.init() to get get information about the specified image. It is called by tmapp.init() if an image name has been specified in the search parameters, but it is also called when a new image is selected in the image browser or when a followed collaborator moves to a new collaboration. Since the situations when a new image is opened can be different and require the function to do different things once the image has been opened, the function includes a parameter for a callback function. For instance, if a viewport position has been specified when calling tmapp.init(), the callback function will tell the viewport to move to the specified position once the image has been loaded.
The first thing that happens when calling the openImage() function is that it checks if any annotations have been placed on the current image. If they have, it asks the user if they actually want to change images, assuming the askAboutSaving parameter is set to true (default is false). If the user rejects, the third parameter of openImage() specifies another callback function that should be called if the user doesn't want to change images.
The function goes on to clear data from the current image before calling the internal function _initOSD, which initiates the OpenSeadragon instance. This includes setting up event handlers, initiating the overlays, and loading the image files. When setting up the event handlers, the callback function mentioned earlier is set to be called after an "open" event has fired.
Annotations are manually placed through the module annotationTool. This module is called both through mouse event handlers specified in tmapp as well as by keyboard shortcuts specified in tmappUI. Internally, this module defines an _activeTool, which is a closure that exposes a number of functions for different tool actions. For instance, clicking in the viewport calls the click() function of the module itself, which in turn calls the click() function of whichever tool is active. If the marker tool is active it creates a new marker, if the polygon tool is active it adds a new corner to the polygon, etc. Once a tool has been used to complete an annotation, it calls the annotationHandler module. The region selection tools also contain calls to the overlayHandler module in order to give the user feedback on the region pending creation.
The annotationHandler module is the central module for handling annotations. It should be called whenever annotations are added, updated, removed or cleared. As mentioned, it is called when the annotationTool module creates a new annotation, but it is also called when a collaborator places an annotation, when annotations are loaded from a file, or when a user first joins a collaboration. This module contains the local canonical state of the currently existing annotations, which is stored in the internal array _annotations. The public functions for this module contain the parameters coordSystem and transmit. The coordSystem specifies which of the three OpenSeadragon coordinate systems the annotation points are being defined with. These are always converted to image coordinates when storing the annotations. The transmit parameter specifies whether or not the annotation action should be transmitted to collaborators. This is used to avoid loops. For example, one collaborator will add an annotation with transmit set to true, which will be broadcast to all other collaborators, causing them to add annotations with transmit set to false.
The representation of annotations in the annotationHandler module does not make any distinction between markers and regions. This difference is only present in the annotationTool and the overlayHandler modules. The annotations contain an array of points used to specify their coordinates. If there is only one point in the array, it is seen as a marker, else it is seen as an annotation. This makes it simpler to work with the annotations in places where the distinction is unimportant.
As the annotationHandler updates the current state of the annotations, it calls the annotationVisuals module in order to give the user a visual representation of the annotations. This module uses the SortableList class to update the list of annotations in the user interface, and also processes annotation using filters created with the filters module. The coordinates listed for the annotations in this list are their centroids. Note that the function used for calculating the centroid of a polygon in the current state of the system does not work properly for self-intersecting polygons. The annotationHandler module also calls the overlayHandler module in order to display annotations in the viewport.
The system can be used to manually save and load annotations locally. There are two important modules for local storage, annotationStorageConversion and localStorage. Both of these are called from the tmappUI module based on user input. The annotationStorageConversion contains two public functions, one for getting an object representation of all currently placed annotations, and one for taking such an object and adding the annotations specified to the current image. This object also contains the name of the image, so that the user can be moved to the right image when loading annotations. The localStorage module is responsible for converting between JSON files and JavaScript objects on the local machine. When saving a file, annotationStorageConversion.getAnnotationStorageData() is first called to get an object, and localStorage.saveJSON() is called to store the object locally. When loading a file, localStorage.loadJSON() is called to get the object, and annotationStorageConversion.addAnnotationStorageData() is called to convert it into annotations.
When users enter CytoBrowser without a collaboration and image specified in the URL, or when they click an image in the image browser, they are prompted to join a collaboration. This is done to ensure that a user is always part of a collaboration when annotating an image. A collaboration is tied to a single image with a unique id, and is used both to allow users to cooperate and to provide persistence of annotation sessions.
Collaboration done through the collabClient module on the client side. Initiating a collaboration is done either through collabClient.createCollab() or collabClient.connect(), the difference being that collabClient.createCollab() first sends a GET request to the server's api/collaboration/id endpoint to get a short, unused id for the collaboration. When connecting, the client opens a WebSocket to the server's /collaboration endpoint. The user can include their currently placed annotations when creating a new collaboration, in which case an empty array called _joinBatch is initialized and stored internally in the module. In the current version of the system, this inclusion only happens if the client temporarily loses its connection, adds some annotations, and regains its connection and rejoins. Before the client can start getting information from the server, it calls the internal function _requestSummary() to get a summary of the current state of the system.
The messages sent over the collaboration are all encoded as JSON, and are handled with the internal function _handleMessage(). The messages all contain a type field to specify which kind of message it is, and the first message a client will receive always has the type "summary". The summary message contains information about what annotations have already been placed, what members are participating in the collaboration, what id the local client is assigned, and what image is being looked at. If the current image is different from the image used in the collaboration, the _joinBatch variable is set to null and another request is sent. Once the client receives a summary message with the same image, the initialization can continue.
If the _joinBatch variable is truthy when the initialization continues, i.e. if it is an empty array, all the locally stored annotations are pushed to it. The messages received in the summary message are then added locally through the annotationHandler module. This is followed by the annotations in _joinBatch being re-added. This ensures that if an annotation from the collaboration and a local collaboration have the same id, the local one will be reassigned before being sent out to other collaborators.
Once the collaboration has been initialized, the collabClient module sends out messages and handles incoming messages. When a local user performs an action that other collaborators should be aware of, such as placing an annotation or moving their viewport, messages are sent over the websocket to other collaborators, and the other collaborators' collabClient modules call the appropriate functions when they receive the messages. In order to move to a collaborator's viewpoint, information sent through memberEvent messages are used to find the right parameters. When following another member's view, the view is simply moved automatically whenever such a message is received.
Clients store the unique id they receive from the server when first connecting to a collaboration, and specify the same id whenever they connect to a new collaboration. This allows the same client to be identified in multiple different collaborations. When one client is following another client and the followed client changes to a new image, they send an "imageSwap" message that specifies which image and collaboration it's moving to. When the following client receives this message they disconnect and connect to the specified collaboration. They also store the id of the followed client in the variable _desiredUser, which is used to re-follow the other client when both clients have reached the new collaboration.
The server side of collaboration is handled in the collaboration module. This module stores one collaboration object per open collaboration. Whenever a client tries to connect to a collaboration that does not exist yet, it is created, and the client is added as a member. The collaboration objects contain information about all the currently stored annotations and all the members connected to the collaboration. One important detail is that each member object contains a ready property, which specifies whether or not they are on the right image and have received the summary. If this is not the case, some messages will not be forwarded to them until they are.
The collaboration module also uses the autosave module to allow for persistence of data. Whenever recent changes have been made to the annotations, a user leaves, or a collaboration is closed, it automatically saves the annotation data in the file system. When a collaboration is reinitialized, be it if the server has restarted or it was automatically shut down due to inactivity, it automatically reloads this data so users can get back to where they left off.
It is possible to revert a collaboration to a previous state. On the client side, this is mainly handled in the versionRevert module, and on the server side, it is mainly handled in the historyTracker module. The historyTracker module uses the diff module to look at what changes have been made between a new version of a file and the current version. The diff module is used to create patches for the changes between each version, and these patches are stored in a list in JSON files prefixed with __HISTORY__. When a call is made to revert a session to an older version, all patches after the specified version are applied. It is possible to change the limit of version entries by changing the value of maxHistoryEntries in the historyTracker module.
The overlayHandler module is used for everything shown in the overlay, including annotations and cursors. A lot of the work is done using d3, which takes care of figuring out which annotations are new, which ones have been removed, which ones are being updated, and so on. This module is called by both annotationVisuals when the annotations are updated, as well by collabClient whenever a collaborator moves their cursor. The module adds mouse tracking to annotations in order to allow things like dragging or right-clicking for comments.
The different elements being displayed in the overlay are dependent on the current state of the viewport. For instance, cursors are always the same size, and the text next to markers are always the right way around. In order to accomplish this, event handlers are added to the OpenSeadragon viewer whenever the viewport is zoomed or rotated, and these handlers call overlayHandler.setOverlayRotation() and overlayHandler.setOverlayScale() in order to properly handle the adjustments to overlay elements.
When the active annotation tool is changed through tmappUI, it also calls overlayHandler.setActiveAnnotationOverlay() to show this to the user. When this function is called for the marker tool, mouse events for regions are disabled, and when it's called for regions, mouse events for markers are disabled. This allows the user to for example place markers inside a region without accidentally dragging the region instead.