Transparent objects brighten when using gammacorrectionshader

[gloriousjob]

Describe the bug

When using sRGB textures (such as in 3DS models), the GammaCorrectionShader seems to brighten the texture more than when the EffectComposer is not used. The GammaCorrectionShader is needed when using the EffectComposer to produce similar colors to when the EffectComposer is not used. I have not been able to upgrade past r111 because of this issue.

As a side note, a similar problem was seen with generic shapes with color but it appears that setting the clearColor on the RenderPass can help with that issue but not with textured objects.

To Reproduce

1. Create a shape with a texture and an opacity of 0.4 and use an EffectComposer, with a GammaCorrectionShader pass.
2. Create a shape with an opacity of 0.4 just using a normal render (no EffectComposer).
3. Note that the colors are brighter on the first shape.

Live example

The following fiddle demonstrates the issue (top is no postprocessing, bottom is using postprocessing):
https://jsfiddle.net/s5tpxq1g/1/

You can see how the bottom is brighter for the texture and a lower opacity makes the issue more apparent.

Expected behavior

Colors match when using GammaCorrectionShader in an EffectComposer compared to not using EffectComposer.

Screenshots
Snapshot from fiddle (top half does not use EffectComposer, bottom half does). The left object shows that a normal object (with clearColor set correctly) does not have an issue but the right object, having textures, differs in brightness.:

Platform:

• Device: Desktop
• OS: Windows 10, 21H2
• Browser: Chrome/Firefox/Edge
• Three.js version: r134 but was noted in r112, when gammaInput and gammaOutput properties were removed and GammaCorrectionShader was recommended for sRGB scenarios.

[WestLangley]

The proper workflow is to render the scene in linear-sRGB color space and convert to sRGB color space as a final post-processing step.

However, currently, three.js converts to sRGB color space in-line in the shader, prior to blending with the drawing buffer.

This is fine if the material is opaque. But if the material is transparent, it is not correct.

I would expect three.js to integrate a proper post-processing workflow into the engine at some point.

/ping @bhouston

[bhouston]

We can also add a post processing step to convert a linear HDR RGB buffer to a LDR gamma corrected buffer. You need at least an FP16 render target for this to work well. It shouldn't really be hard -- I think all the code is in Three.js. This is actually how Clara.io worked back in 2014:

https://clara.io/view/9e2c46ce-6ada-450c-8acd-c44b84de2a38

I guess when you say we need to integrate a proper post-processing workflow, @WestLangley, you are meaning out of the box. That makes sense now.

I actually switched from the proper HDR FP16/FP32 render buffer approach in 2016, and I contributed that back to Three.js here #8250, because many mobile devices simply didn't have enough memory to handle it properly, especially Apple iPads of the era. The approach we switched to is what Three.js currently uses as the default workflow -- it was actually awesome and fast. It does have a few limitations though as pointed out in this bug report.

Thus we could go back to this more traditional approach now that mobile hardware has caught up.

[bhouston]

And interesting @WestLangley and I had this exact same discussion back then: #8250 (comment)

[Mugen87]

So to clarify, the upper part of the fiddle is actually too dark whereas the lower part (using post-processing) is the correct image.

[bhouston]

@Mugen87 did you delete the comment where you mentioned the framebuffer/texture format "SRGB8_ALPHA8_EXT"? Could we use this as the render target format? From a quick reading, assuming the browser does the blending correctly between two separate sRGB8_ALPHA8_EXT buffers, it would solve the issue no?

[Mugen87]

Well, I was not sure about the details of my comment so I've deleted it^^.

I think the goal should be to avoid all decodes/encodes and tone mapping in the material shaders. Avoiding the decodes should be easier to implement via changes like #23039.

As mentioned in this thread, avoiding the output encoding and tone mapping in the materials means the renderer has to implement some sort of post-processing. I'm not yet sure how this would look like though (implementation-wise) .

[looeee]

Note this has been brought up a few times over on the forum as well, so the current behavior does confuse people.

The difference is also stronger the more transparent an object is - so there's not much difference with opacity = 0.9 but with opacity = 0.1 it's quite obvious.

[gloriousjob]

Not sure if I missed something but I tried updating the fiddle to use UnsignedByte, which should have enabled SRGB8_ALPHA but it seems to have no difference:
https://jsfiddle.net/2qjth7un/

So to clarify, the upper part of the fiddle is actually too dark whereas the lower part (using post-processing) is the correct image.

So when I've used things like a transparent vase, the post-processing makes it look like there's a light brightening the vase and makes seeing through the vase difficult so I think I'd actually prefer the upper picture, although I can see how the example I gave might look better.

[Mugen87]

From a quick reading, assuming the browser does the blending correctly between two separate sRGB8_ALPHA8_EXT buffers, it would solve the issue no?

Yes. And according to KhronosGroup/WebGL#2474 (comment), blending will indeed work correctly when using sRGB8_ALPHA8.

However, I want to make clear that WebGL does not allow to define the color space of the default framebuffer. That means we have to perform the final linear-sRGB encode ourself in a custom pass.

I've hacked around and verified that the components of the framebuffer's color attachment are actually gl.SRGB when a render target has the RGBA8 format + sRGBEncoding. You can see this by checking the console output and the respective code in WebGLTextures in this demo:

https://raw.githack.com/Mugen87/three.js/dev19/examples/webgl_geometry_cube.html

As you can see, the scene is too dark since I have removed the encodings_fragment chunk from MeshBasicMaterial. Meaning no sRGB encode happens in the material shader anymore.

[Mugen87]

@Kangz Is it possible with WebGPU to configure the canvas context with a sRGB format? We are currently using bgra8unorm but when trying bgra8unorm-srgb nothing is rendered and the following warning appears in the browser console:

unsupported swap chain format
context configuration is invalid.

Tested with Chrome Canary 99.0.4819.0 on macOS.

According to the spec bgra8unorm-srgb should be supported. Does it mean this part is not yet implemented?

The idea is to write linear color values into the buffer but the format automatically performs a linear to sRGB encoding. Like mentioned in #23019 (comment), it's not possible to configure the drawing buffer's color space in WebGL.