Enhancement: add renderer.outputEncoding

[WestLangley]

As discussed in #11332

1. renderer.gammaInput needs to be deprecated. We now support texture.encoding instead.

2. renderer.gammaOutput needs to be deprecated and replaced with renderer.outputEncoding when rendering to the screen. We already support renderTarget.texture.encoding when rendering to a render target.

3. WebGLPrograms.getProgramCode() needs to be updated to respond to changes in renderer.outputEncoding. Setting material.needsUpdate = true will then force the change.

4. Legacy warnings need to be added, and the examples need to be updated to accommodate this change.

[WestLangley]

Actually, before we go down that path, perhaps we should decide on decoupling "encoding" into color space and encoding format, as suggested by @bhouston.

If we split color space conversion from the encodings, we could do the above. It would be more flexible and clear. We could also support LUTs in the color space conversions as well.

And then each texture and render target should have both a ColorSpace (Linear, sRGB, AdobeRGB, Gamma20, Gamma22) and a Encoding (None, RGBE, RGBM, RGBD, LogLUV, CMYK) parameter.

This is something I would support.

[bhouston]

Actually, before we go down that path, perhaps we should decide on decoupling "encoding" into color space and encoding format, as suggested by @bhouston.

Separating colorSpace and encoding is not that hard. I would just copy the code used to inject functions for encoding into the glsl and modify it to inject colorSpace transforms. And then split the enums into a color space list and an encoding list. I guess I am saying just use the existing design for encodings, but duplicate it again for color spaces.

I think that is sort of the easy, obvious solution.

[bhouston]

The injectors are pretty simple - they just create specially named functions that map onto the chosen encoding function:

https://github.com/mrdoob/three.js/blob/dev/src/renderers/webgl/WebGLProgram.js#L12

I use the same injection idea for tonemapping:

https://github.com/mrdoob/three.js/blob/dev/src/renderers/webgl/WebGLProgram.js#L51

[bhouston]

Hmm... I wonder if tone mapping could be combined with color spaces? They are sort of the same thing theoretically -- a color transform. And they sort of happen at the same time in the renderer pipeline (before the encoding.)

And then moved from a global on WebGLRenderer to something that is texture specific (except we would use the global colorSpace / toneMapping operator when one doesn't have a render target, in the same fashion that WebGLRenderer.outputEncoding would be used.)

I have run into issues with the global tone mapping operator -- and as it is currently designed it is annoying to disable and re-enable. And I think a per texture colorspace/tonemapper may be more useful, and impose no limitations that I can think of.

Is there precedent for considering tonemapping a form of color space conversion? I wonder because it would justify combing them.

[bhouston]

I think that the order maybe:

toneMapping (gamut reduction)
colorSpace (gamut transforms, assumes inputs are between 0-1)
encoding (assumes inputs are between 0-1)

I guess you may want to use a Reinhart tone map operator may or may not want sRGB output -- thus that argues for a separation of toneMapping from colorSpace.

Another complicating is that HDR TVs and Monitors are coming and use different gamma curves such as this one: https://en.wikipedia.org/wiki/Rec._2020

[WestLangley]

To summarize:

toneMapping ( to LDR )
colorSpace ( Linear, sRGB, AdobeRGB, Gamma20, Gamma22, Grayscale, CustomLookup )
encoding ( RGBA, RGBE, RGBM, RGBD, LogLUV, CMYK )

This means all textures and render-target textures have the following two properties:

texture.colorSpace = THREE.sRGBColorSpace; // for example
texture.encoding = THREE.NoEncoding; // is THREE.RGBAEncoding a better term ???

So instead of adding renderer.outputEncoding, we would add:

renderer.colorSpace = THREE.sRGBColorSpace; // THREE.LinearColorSpace, etc..
// renderer.encoding = THREE.RGBAEncoding; // I guess that is not needed

that argues for a separation of toneMapping from colorSpace.

Yes.

[bhouston]

Yeah. Just to be clear I see this as an eventual end point:

texture.toneMapping
texture.colorSpace
texture.encoding

// and for the case of the null renderTarget, rendering to screen (which is sort of an annoying exception):
renderer.toneMapping
renderer.colorSpace
// renderer.encoding - this should not need to be set, always RGBAEncoding or NoEncoding.

I guess putting these colorSpace/toneMapping/encodings on textures/renderTargets could get weird when one reuses them in the EffectComposer -- you need to be careful what you set because we recycle readBuffer, writeBuffer as we step through the composer. I guess we already have that problem, so maybe it makes no real issue right now.

[bhouston]

Actually you are right, no encoding needed for renderer. It should always be RGBAEncoding.

[WestLangley]

texture.toneMapping does not sound right to me...

I am now thinking we need to separate further:

toneMapping ( HDR to LDR )
colorMapping ( to Grayscale, to CustomLookupTable )
colorSpace ( Linear, sRGB, AdobeRGB, Gamma20, Gamma22 )
encoding ( RGBA, RGBE, RGBM, RGBD, LogLUV, CMYK )

Tone mapping in this case is really a property of a render target. The renderer is a stand-in for the default render target:

renderer.toneMapping (when rendering to the screen)
renderTarget.toneMapping

Likewise, color mapping: it is essentially a post-processing effect, but applied as-we-go.

renderer.colorMapping (when rendering to the screen)
renderTarget.colorMapping

Color space is a property of a texture, render target texture, and renderer output.

texture.colorSpace
renderTarget.texture.colorSpace
renderer.colorSpace

Encoding is a property of a texture and a render target texture.

texture.encoding
renderTarget.texture.encoding

So, to summarize

renderer.toneMapping
renderer.colorMapping
renderer.colorSpace
//renderer.encoding - not needed; always RGBA

renderTarget.toneMapping
renderTarget.colorMapping

texture.colorSpace
texture.encoding

That's the way I see it... for now. :)

[bhouston]

Well, we could view this as a list of inline post processing transforms. Encoding we run through the set forward with forward transforms, decoding we go backwards with inverse transforms.

Such an interface would look like:

texture.texelOps.add( THREE.FilmicToneMapping, { exposure: exposure, whitePoint: whitePoint } );
texture.texelOps.add( THREE.sRGBColorSpace );
texture.texelOps.add( THREE.RGBMEncoding, { multiplier: 16 } );

This reduces it to a single list of transforms/operators, and you can have an arbitrary number. Implementation would be simpler and it would be more flexible going forward. Maybe each operator could be a class that contains its own code injection method so it is better encapsulated and infinitely extensible by others -- rather than hard coded enums on THREE.

I've never seen this design paradigm before. That doesn't mean it is wrong, but it makes me not super confident in pushing it. But if others agree....

[bhouston]

The texel op class would look like this -- this is the default that doesn't do anything. You would override it with better data.

class TexelOp {
   fowardOp: function( name, parameters ) {
      return "vec4 " + name + "( vec4 input ) { return input; }";
   }
   backwardOp: function( name, parameters ) {
      return "vec4 " + name + "( vec4 input ) { return input; }";
   }
}

[mrdoob]

@bhouston I like that approach!

[donmccurdy]

It may help avoid surprises for new users if we changed the default renderer.gammaOutput=true, or whatever the equivalent outputEncoding or texelOp syntax ends up being... Is it worthwhile to do that, and deprecate gammaInput, before we implement a more flexible interface here?

GLTFLoader automatically sets material.map.encoding = THREE.sRGBEncoding (required by glTF, and likely to be generally correct for other formats) and assumes the user has set gammaOutput=true. Frequently, users find this confusing.