Color Library Design - Color Models

[dragonfly-ai]

With respect to definitions written in Color Library Design - What to Model?, let's explore tradeoffs involved in color model implementation. This post begins the discussion with a kind of type safety concern, but the thread should definitely branch into accessibility, memory footprint, and performance.

Programmers encounter Color Models much more often than any other Color Science concept, so any color library had better maximize performance and accessibility at this level.

Lots of implementations of RGB, RGB24, ARGB32, RGBA32, HSL, HSV, CMY, CMYK, etc already exist in all languages, however each design I've seen only addresses time and space tradeoffs: e.g. class RGB { float red; float green; float blue; float alpha; } for convenience vs class {int rgba;} for speed and compactness. Whether the color itself comes from sRGB, Apple RGB, or Display P3, libraries do not know and most programmers wouldn't know to ask, even though in applications such as color comparison, the color space that defines the RGB values matters profoundly! Shouldn't a color library protect users from accidentally training the stop sign detector on sRGB images, but then deploys to iPhone apps with cameras that sample in Display P3?

How could a library guard inexperienced users from the consequences of cross-color space comparisons?

Some options may include:

Data Approaches:

• Reference: Should every RGB value carry a reference to its Working Space, Gamut, and/or Color Space? In C family languages, that implies an O(4) X memory footprint penalty incurred merely by adding 4 extra pointers.
• Parameter: Should every comparison method/function in the RGB Color Model require a parameter for the Color Space of each color to make sure they match?
• Context/Configuration/Environment: Should all operations occur in a context defined by the WorkingSpace, Gamut, and Color Space? How could one manage multiple contexts and interactions between them?

All of these might protect developers from blundering into color comparisons across color spaces, however they seem prone to increase obtuseness with a proliferation of if statements:if c1.colorSpace == c2.colorSpace: and also increase memory footprint. On the other hand, it supports user defined types for Working Spaces, Color Spaces, Gamuts, and Color Models fairly straight forward, but would a programmer tolerate a library that preconditions all functionality on the developer's ability to select the correct working space? Done well, it could educate developers, but done poorly, it will alienate them.

Type Approaches:

• Type: Instead of providing an RGB type, provide Adobe_RGB_1998, sRGB, Display_P3_RGB. Naming conflicts abound when we model the Color Spaces with the same names.
• Type Parameter: Should the RGB class take type parameters in the style of C++ templates or Java Generics? e.g. RGB[Display_P3]. If so, how does a Working Space (with a structure regular enough to be a class itself) differentiate itself at the type level if it's only meaningful distinction comes through its values? In other words, Working Space A is only different from Working Space B if it has a different Illuminant, chromatic primary, or transfer function. Does it make any sense (in code) to ascribe a unique type to each WorkingSpace?

Addressing the issue from the type level eliminates all of the obtuseness and bloat. In most cases involving only one color space, developers can treat the library like any other:

from colorio.sRGB import *
c1 = RGB(1.0, 0.5, 0.0)
c2 = RGB(0.3, 0.75, 0.0)
c1.similarity(c2)

Then, in relatively rare but meaningful cases like the stop sign example:

import colorio.sRGB #sRGB Working Space
import colorio.Display_P3 #Display_P3 Working Space
c1 = sRGB.RGB(1.0, 0.5, 0.0)
c2 = Display_P3.RGB(0.3, 0.75, 0.0)
c1.similarity(c2)  # ERROR!

However, as mentioned, this implies that a type level conception of the Working Space which, mathematically, varies only in data but not structure. It also seems to exclude the possibility of Color Spaces defined at runtime.

How else might a color library capture the relationship between Color Spaces and Color Models?

[dragonfly-ai]

Typed vs Untyped Color Models.

A huge majority of Color Models take the form of three dimensional vectors with bounded, real valued, components. Should a library make use of a type system to distinguish between RGB, Lab*, HSV, XYZ, etc, or should it strive for the simplicity and performance made possible by a common 3D Vector type?

Typed Color Models can prevent developers from accidentally treating an instance of RGB as an XYZ and save them a lot of time debugging the subtle errors that come from these mistakes. However, types introduce performance penalties and bloat as every conversion operation may reference different constructors or add arguably extraneous instance variables to the heap. Typed Color Models also make it more difficult for a library to support User Defined Types, or runtime defined types.

Untyped color models, using a common type for all, can significantly decrease memory footprint, compute times, and friction for introducing user defined color models at compile time or runtime. However, they ultimately push the responsibility of type consistency onto some kind of contextual mechanism and/or increased developer cognitive load.

How would you balance these tradeoffs?

Would you prioritize type safety, sheer performance, or extensibility? On that last note, should a color library try to support user defined color types at all; if so, should it support them at compile time, run time, or both?

[dragonfly-ai]

Another crucial and commonly addressed aspect of color models involves the conversions between them, e.g.:

RGB <-> HSV
RGB <-> ARGB32
RGB <-> XYZ <-> Lab*
RGB <-> CMYK

In each case, a color library must define a high performance function, or composition of functions, and make it accessible in various code contexts, but each design option has consequences:

Object Oriented - Methods and Dot Access:
A class based, forward only scheme assumes that each color model class includes logic to convert to every other color model class. One imagines a general ColorModel interface that declares a forward conversion method for every supported color model in the library. Such a library implicitly represents the space of color conversions as a directed graph with vertices defined as color model class definitions and edges defined by methods within those classes. For example:

rgb1 = RGB(0.75, 0.75, 0.1)
hsv1 = rgb1.to_HSV()
rgb2 = hsv1.to_RGB()

# Syntax variants:
c = rgb1.hsv() # accessor
c = rgb1.asHSV() # explicit 1
c = rgb1.toHSV() # explicit 2
c = rgb1.getHSV()  # property/getter

Alternatively, a library could take a backwards only, singleton oriented approach:

rgb1 = RGB(0.75, 0.75, 0.1)
hsv1 = HSV.from_RGB(rgb1)
rgb2 = RGB.from_HSV(hsv1)

# Syntax variants:
c = RGB(hsv1) # constructor
c = RGB.hsv(hsv1) # factory

Further, one could model the color conversion in an undirected way by referencing conversion methods from both class instances and singleton objects. Generally, to reduce redundancy, the singleton can define the conversion function while the class can reference that definition, or vice versa. This to/from syntax might maximize intuitive convenience for users of the library, but at costs:

1. Burdens every color model with the responsibility of understanding every other in the conversion graph.
2. Significantly increases friction when adding more color models, as each new model requires an edit to every existing model.
3. Sacrifices any possibility of support for user defined color models, let alone those defined at runtime.
4. Only supports library architectures with typed color models.

Procedural Approaches
External functions could independently provide all of the conversions:

def RGB_HSV(rgb):
def HSV_RGB(hsv):

# Syntax Variants:
def rgb2hsv(rgb):
def RGBtoHSV(rgb):
def RGB2HSV(rgb):
def RGB_to_HSV(rgb):

This method promises high performance and supports both typed and untyped color models, but seems error prone for its lack of modularity. Without conforming to any interface, the library maintainer must resort to some external means of keeping track of whether every edge in the conversion graph has a corresponding conversion function.

Transformer Classes or Lambdas
A library with a typed color model architecture could support user defined color types by moving conversion functionality out of singleton objects and classes into lambdas or transformers.

class ColorTransform:
def __init__(self, conversion):
self.convert = conversion

class RGB2HSV(ColorTransform):
def __init__(self):
  conversion = lambda rgb:
    # conversion code.
  ColorTransform.__init__(self, conversion)

This can support limited runtime types as well, through instances of ColorTransform invoked directly with some runtime defined conversion lambda. The main limitation comes from an inability to discern equivalence between two independently instantiated run time defined types.

What other broad ways could a color library expose color model conversion functionality?