Roadmap for a color-managed workflow in three.js

[donmccurdy]

Overview

Summary issue to organize and track progress toward offering a color-managed workflow in three.js. Loosely based on Maya's documentation, I'll define color management as follows:

Color management involves applying the appropriate transforms to convert between color spaces as needed. These transforms are applied at specific points along the initialization, rendering, and display processes. For example:

• On input, a transform is applied to convert colors and textures from the color space in which they were saved into the working color space.
• As you work, colors in the working color space may be converted to/from other color spaces in order to accommodate bit depth and precision considerations.
• On output to an image or display, an output transform is applied to convert colors appropriately for the image format or browser and device capabilities.

Roadmap

1. (1.1) Add documentation of color management APIs (@donmccurdy)
   • Color Management: Documentation #23430
2. (1.2) Loaders identify and convert color spaces correctly (@gkjohnson)
   • Loaders: Ensure all loaders properly label and convert color textures and colors to sRGB / Linear correctly #23283
3. (1.3) Rename "encoding" properties more precisely as "colorSpace" (@donmccurdy + @Mugen87)
   • renderer.outputEncoding → renderer.outputColorSpace
     • WebGLRenderer: Rename outputEncoding to outputColorSpace. #23936
     • WebGLRenderer: Replace .outputEncoding with .outputColorSpace #25756
   • texture.encoding → texture.colorSpace
     • Texture+WebGLRenderTarget: Replace .encoding with .colorSpace #25771
4. (1.4) Change output color space default from Linear-sRGB to sRGB (@donmccurdy)
   • WebGLRenderer: Default output to sRGB #25783
5. (1.5) Convert some inputs automatically from sRGB to Linear-sRGB (@donmccurdy)
   • hexadecimal and CSS inputs: Color: Add 'colorSpace' argument for getters/setters #23392
6. (1.6) Change texture.colorSpace default to THREE.NoColorSpace (e.g. normal maps)
   • Texture+WebGLRenderTarget: Replace .encoding with .colorSpace #25771
7. (1.7) Ensure intermediate frame buffers have sufficient bit depth for their color space. See outputEncoding not handled correctly when using sRGB render target in WebGL2 #23251 (comment). No changes required.
8. (1.8) Let CubeTextureLoader return sRGB cube textures by default.
   • CubeTextureLoader: Return sRGB textures by default. #26162
9. (1.9) Change renderer.useLegacyLights default to false
   • WebGLRenderer: Deprecate useLegacyLights, change default to false. #26392

Timing considerations: If we are doing (1.3), (1.4), and (1.5), it would cause less disruption to existing code if we can make these changes within the same release. Fewer changes, if any, will be required for end-user code.

Future

These changes are more speculative, and may be considered explorative steps toward support for wide-gamut color spaces in WebGPU and SVG renderers.

1. (2.1) Support changing ColorManagement.workingColorSpace
2. (2.2) Support color input/output in Display P3 color space
3. (3.3) Support vertex color workflow changes if necessary (ColorBufferAttribute?)

Continued in #26479.

[donmccurdy]

/cc @bhouston @gkjohnson @WestLangley I'm opening this is a higher-level tracking issue, since (a) any particular PR may be closed and lose that discussion context, and (b) more changes are needed than just #23392.

[gkjohnson]

(1.2) Loaders identify and convert color spaces correctly (@gkjohnson)

From my perspective this is nearly done. Aside from FBX the most common format loaders and exporters are converted to handle the current three.js color system correctly. The rest I think are less commonly used and / or I don't know a lot about. It would be nice if someone could help with the FBXLoader and example conversion.

[donmccurdy]

#23392 (1.5) has been merged, and I'm hoping #23430 (1.1) will also make it into r139.

At this point we may want to pause for a release or two, and see if there's feedback from users adopting the new ColorManagement.legacyMode = false option. I'm inclined to batch the 'breaking' changes together as much as possible. Those include (1.3) and (1.4) above, and — perhaps depending on feedback? — enabling .legacyMode = false by default.

[donmccurdy]

Not going to push too hard if others feel this is out of scope, but I wonder if we should consider enabling renderer.physicallyCorrectLights = true by default at some point as well.

[bhouston]

we should consider enabling renderer.physicallyCorrectLights = true by default at some point as well.

That would be amazing! It was always a hack to make it false, but it was required initially until we settled on it being the right solution: #8260

[donmccurdy]

Feedback from the R3F folks (pmndrs/react-three-fiber#2127) has been that switching old demos to physicallyCorrectLights is hard, e.g. every light needs .intensity *= Math.PI and a considerable reduction in .decay to preserve the original look.

What is WebGPURenderer doing in terms of physicallyCorrectLights? I can't tell from the code but I think we should at least omit the property there and stick with physically-correct lighting. Users can always reduce light.decay if they need more artistic control.

[drcmda]

when we switched to physicallyCorrectLights it seemed a change that is too big to just put out there. it didn't seem straight forward to restore projects and demos, everything turned pitch black. two properties (decay and intensity) didn't make it easier to comprehend how to fix it. when three makes it the default i think it should first educate people how to use it properly, similar to colormanagement.legacy=false and the new article explaining everything.

as for color management, react+three + eco system uses it by default now. i have already updated most sandboxes and everything works great. i love being able to omit the occasional convertSRGBToLinear and everything behaves much more consistent.

[mrdoob]

What we could do is to make WebGPURenderer physically correct and be the only mode.

If a developer uses WebGPURenderer and wants to use WebGLRenderer as fallback, they'd have to make sure they're setting physicallyCorrectLights = true in WebGLRenderer.

However, I do not know how to change the lights decay to a physically correct value by default without breaking things.

/cc @sunag @Mugen87

[donmccurdy]

I don't feel strongly on whether the decay default really needs to change. It's true that it's only physically correct with a value of decay=2, but given all the other constraints of lighting in WebGL that's not a hill I'd choose to die on. And certainly there's no "one size fits all" upgrade path to get people there.

Perhaps we could change the decay default only under physicallyCorrectLights somehow; I don't think most users enable that setting so a breaking change in that mode might not be so bad, and e.g. glTF files containing lights will already set decay=2.

[Mugen87]

I'm glad this discussion came up^^. I have not added physicallyCorrectLights on purpose when filing the initial version of WebGPURenderer. #23614 (comment) sounds good to me.