Examples: webgl_loader_gltf_transmission issues

[WestLangley]

Reposting #21000 (comment) so as to consolidate some remaining issues.

1. webgl_loader_gltf_transmission has the transparent property of each sphere to set to false. Is that what the loader should be doing? What should transparent be set to when transmission is non-zero?

2. After setting transparent to true, only front-faces show the hot-spot. (This may be a known limitation, but @mrdoob just added support for properly rendering back-faces of double-sided meshes.) There should be two hot-spots.

3. FIXED: This is caused by issue 7. below.

4. FIXED Also, in Chrome and Firefox (not Safari)

THREE.WebGLProgram: gl.getProgramInfoLog() WARNING: Output of vertex shader 'vUv' not read by fragment shader

Transmission always sets the USE_UV flag.

5. FIXED: If performant, I think the dimensions of the "transmission render target" should match the dimensions of the "current render target". Doing so will prevent artifacts like the following:

6. FIXED In the following code snippet, transmissionFactor is applied to getIBLVolumeRefraction() twice. I expect that is a physical modeling error.

three.js/src/renderers/shaders/ShaderChunk/transmission_fragment.glsl.js

Lines 23 to 28 in dd6ab3c

	vec3 transmission = transmissionFactor * getIBLVolumeRefraction(
	normal, v, roughnessFactor, material.diffuseColor, totalSpecular,
	pos, modelMatrix, viewMatrix, projectionMatrix, ior, thicknessFactor,
	attenuationColor, attenuationDistance );
	totalDiffuse = mix( totalDiffuse, transmission, transmissionFactor );

7. FIXED Also in the above snippet, totalSpecular has units of radiance, and therefore cannot be the correct argument because the function expects a unit-less quantity. A reasonable guess would be material.specularColor, instead.

• Device: Desktop
• OS: macOS 11.4
• Browser: Chrome, Firefox, Safari
• Three.js version: r130dev

/ping @takahirox @donmccurdy

[donmccurdy]

I think that displaying transmissive materials (as defined by KHR_materials_transmission) through other transmissive materials may be considered a feature request. Displaying opaque materials and the environment map through the transmission — as we do now — was priority (1), displaying alpha blend materials through transmission is probably second (2), and transmission through transmission third (3).

Similarly, I believe I'm seeing (1) in https://sandbox.babylonjs.com/ and https://github.khronos.org/glTF-Sample-Viewer-Release/, with reflections only from the nearest transmission material. Adding (2) and (3) requires techniques like depth peeling; see comment by @MiiBond in #16977 (comment). This would certainly be a welcome addition, but we'd want the flexibility to configure the number of layers, or disable that entirely for performance.

[mrdoob]

4. Also, in Chrome and Firefox (not Safari)

THREE.WebGLProgram: gl.getProgramInfoLog() WARNING: Output of vertex shader 'vUv' not read by fragment shader

Transmission always sets the USE_UV flag.

Fixed ac6d30a

[mrdoob]

5. If performant, I think the dimensions of the "transmission render target" should match the dimensions of the "current render target". Doing so will prevent artifacts like the following:

Worth noting that this would only work in WebGL2 as we do need mipmaps.

[mrdoob]

Another problem with the current transmission code is that it "breaks" in transparent contexts.

Here I'm setting alpha: true and renderer.domElement.backgroundColor = 'red':

/cc @elalish

[elalish]

I consider this pretty important. The DOM uses transparency pretty extensively which is why model-viewer uses alpha: true. This way DOM backgrounds and gradients and such fit nicely into the scene and the 3D doesn't have to look like a rectangle. Of course this is non-physical, so we'll need some kind of simple approximation. I'm hoping we can just get the shader to set the alpha to whatever the transmission % of the pixel was, though we'll have to be careful to keep it opaque if it was already drawn to by something opaque. It won't blur the DOM background, but I think that's fine, and more or less in line with not seeing transparent objects through each other.

[gonnavis]

Hello @takahirox, Thanks for the PR! I found that current example has gaps if look from a side view or back view:

But the front view is OK.
I tried with a simple box, seems all refracted pixels are pushed to +Z axis a little?

[mrdoob]

I tried with a simple box, seems all refracted pixel are pushed to +Z axis a little?

@takahirox Any ideas?

[takahirox]

Unfortunately I currently have lack of time to tackle this problem.

I tried with a simple box, seems all refracted pixel are pushed to +Z axis a little?

This guess sounds right to me. The ray is calculated in

src/renderers/shaders/ShaderChunk/transmission_pars_fragment.glsl.js

I'm happy if anyone can help the investigation in the details.

Update: The model looks fine if thickness is zero. So I suspect there is a problem in this function or the parameters passed to the function can be wrong.