https://github.com/hpotechius/ColorTransferLib

ColorTransferLabV2

ColorTransferLabV2 is a web-based user interface for the application of color transfer, style transfer, and colorization algorithms on different data types, including Images, Videos, Point Clouds, Meshes, Light Fields, Volumetric Videos, and Gaussian Splattings, with the possibility to visualize these data types. Additionally, 20 evaluation metrics are included to assess color transfer results. It is based on our previous tool ColorTransferLab.

This tool is based on WebRTC communication between the client, which is the User Interface, and the compute node, which applies computations using the Python library ColorTransferLib. This library contains available algorithms and image quality assessment metrics.

1. System Architecture

This system consists of four components: Client, Web Server, Signaling Server, and Compute Node, which will be explained in the following sections.

1.1 Client and Web Server

The client is the user interface, served by a web server available at https://potechius.com/ColorTransferLab or self-hosted. It allows users to visualize data and connect to an available Compute Node to apply the aforementioned algorithms.

1.2 Signaling Server

To connect two instances, such as the client and compute node, within this WebRTC-based setup, the signaling server announces them to each other. The server is available at https://signal.potechius.com and also displays a list of available instances when accessed via a browser.

1.3 Compute Node

This instance must be provided by the user (see Section 2: Usage). It supplies the necessary data and computational power to apply the algorithms. It is a Python instance with an installed version of ColorTransferLib.

The above image illustrates the three stages of the compute node:

• Initial Stage: The node is only connected to the signaling server, which can also be provided by the user.
• Waiting for Connection: After clicking the "Connect to Signaling Server" button, the compute node registers with the signaling server and enters the Idle state. The compute node now awaits a client connection to begin processing.
• Connected Stage: Client and compute node are connected for peer-to-peer data transmission.

2. Setup

In this setup, the signaling server is provided via https://signal.potechius.com, and the user interface is served through the web server at https://potechius.com/ColorTransferLab. The only instance that must be provided by the user is the compute node. Notably, both the signaling server and the web server are included in this repository and can be self-hosted. The following steps must be followed to run the compute node and test the application.

2.1 Advanced Setup

1. Clone the repository and go into the directory

   git clone git@github.com:hpotechius/ColorTransferLab.git
   cd ColorTransferLab
   

2. Create and activate an environment env using Python3.10

   python3.10 -m venv env
   source env/bin/activate
   

3. Install the necessary packages

   pip install -r ressources/requirements/requirements.txt
   

4. Run the Compute Node

   cd instances/ComputeNode
   python main_computenode.py
   

   • During the first run, the Compute Node will download the dataset from https://potechius.com/Downloads/Datasets/ColorTransferLab_Database.zip.
   • The Compute Node will automatically verify whether the specified signaling server is online.
5. Connect to Signal Server
   • Press the "Connect to Signal Server" button (https://signal.potechius.com).
   • The Compute Node will now enter a waiting state until a client connects to it.
6. Connect to Compute Node
   • Open https://potechius.com/ColorTransferLab in any browser.
   • The Compute Node should now appear in the Compute Node section of the web interface. Press the button to view all available algorithms and data.

2.2 Easy Setup

1. Run the script to be guided through the installation process.

   cd ressources/scripts
   chmod +x runComputeNode.sh
   ./runComputeNode.sh
   

2. After running the script for the first time, all installation steps can be skipped by using the --run parameter.

   ./runComputeNode.sh --run
   

2.3 Notes

• Due to the WebRTC-based system architecture, the client and compute node do not need to run on the same system.
• Depending on the network environment, a connection may not always be established using a STUN server. TURN servers are not provided.

3. Datatypes

4. Interface

ALGORITHMS SIDEBAR	DESCRIPTION
Color Transfer	This tab provides a collection of 11 color transfer algorithms..
Style Transfer	This tab provides a collection of 5 style transfer algorithms.
Colorization	This tab provides a collection of 3 colorization algorithms.

RENDERER AREA	DESCRIPTION
Source	This area can visualize images and 3D point clouds by dragging it from the Items-Menu and dropping it to this area. The object within this area will be used as *Source*-Input for the color transfer.
Reference
Single Input	This area can visualize images and 3D point clouds by dragging it from the Items-Menu and dropping it to this area. The object within this area will be used as *Reference*-Input for the color transfer.
Color Palette	This area allows the user to select multiple colors which will be used as reference.
Output	his area can visualize images and 3D point clouds applying a color transfer algorithm. Only the results can be visualized here. See `Buttons/Start`-Section for more information.

DATA SIDEBAR	DESCRIPTION
Database	Contains a folder structure of available data.
Output	Contains the results which are created by the user.
Uploads	Images which are uploaded, will be available here. See `Buttons/Upload`-Section for more information.
Items	Shows the folders and objects which are contained within the corresponding Database-Folder.

CONSOLE TAB	DESCRIPTION
Console	Shows information about the current state of the application with corresponding time stamps.
Evaluation	Provides the user with information about objective evaluation metrics after clicking the *Evaluation*-button. See `Buttons/Evaluation`-Section for more information.
Configuration	Configurable parameters will be shown after clicking a chosen color transfer algorithm
Color Statistics	Shows the 2D color histograms for source, reference and output with the respective means and standard deviations.
Information	This area shows general information of the clicked color transfer algorithms.

BUTTONS	DESCRIPTION
Upload	By clicking on this button the user can select an image which will be uploaded to the database. The uploaded object will be availabe in the `Items`-Area after clicking the `Uploads`-Button in the `Database`-Area.
Evaluation	By clicking this button, the evaluation of the output against the source and reference will be performed. This feature is only available for image-to-image color transfer.
Start	By clicking on this button the user starts the color transfer process. The resulting object will be displayed in the `Output`-Renderer. This only works if both a source and a reference object are selected and a color transfer algorithm was chosen.

SETTINGS	DESCRIPTION
Single View	Toggles between the Single View of the renderer area and the simultaneous view of all three renderers — source, reference, and output.

5. List of other Color Transfer Tools

• Palette-based Photo Recoloring
• L2 Divergence for robust colour transfer: Demo
• A Web App Implementation for Image Colour Transfer
• Photo Recoloring
• Adobe Photoshop's Color Transfer neural filter

6. Acknowledgements

• The light field renderer is adapted from hypothete's lightfield-webgl2 repository.
• Gaussian Splatting renderer is adapted from mkkellogg's GaussianSplats3D repository

7. Citation

If you utilize this code in your research, kindly provide a citation:

@inproceeding{potechius2023,
  title={A software test bed for sharing and evaluating color transfer algorithms for images and 3D objects},
  author={Herbert Potechius, Thomas Sikora, Gunasekaran Raja, Sebastian Knorr},
  year={2023},
  booktitle={European Conference on Visual Media Production (CVMP)},
  doi={10.1145/3626495.3626509}
}