GLTF with material that uses `transmission` misses color in model-viewer but not in other renderers like babylon.js

[Bersaelor]

Description

When I preview our model using modelviewer the Plastic.002 material is missing it's color.
Other renderers like bablyonjs look fine.

I have been trying the new transmission property, so somehow the different parameters between Plastic.001 (which just has alpha < 1) and the new way of achieving transparency using the transmission parameter remove the color from the material.

Live Demo

Drag the model into the above mentioned viewers and it will look like this:

Version

• model-viewer: v2.1.1

Browser Affected

• Chrome, version: 108.0.5359.124
• Edge
• Firefox
• IE
• Safari

OS

• Android

• iOS

• Linux

• MacOS

• Windows

[elalish]

Hmm, the difference seems to be:

      "alphaMode": "OPAQUE",
      "doubleSided": true,
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 1
        }
      },
      "name": "Plastic.002",

vs

      "alphaMode": "BLEND",
      "doubleSided": true,
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 0.5
        }
      },
      "name": "Plastic.001",

I'm not quite sure how one should interpret "transmissionFactor": 1 together with "alphaMode": "OPAQUE". But even when I set it to BLEND, it still looks wrong (no color); might help to reduce the transmissionFactor. In any case, this is a three.js rendering issue, so I'd recommend you file an issue there are get a discussion going.

[Bersaelor]

Thank you for your reply @elalish!

So the last weeks I've been using transmission: 1 with three.js version "^0.146.0" in our plain Three.js app with no problem. There I setup the THREE.MeshPhysicalMaterial manually , e.g.

    var material: THREE.MeshPhysicalMaterial = new THREE.MeshPhysicalMaterial({
        opacity: opacity,
        color: Number(hex), 
        reflectivity: m.parameters.reflectivity,
        metalness: m.parameters.metallicity,
        roughness: m.parameters.roughness,
        transmission: m.parameters.transmission
    });

and never had any issues with the colors.

The materials can be fine-tuned with the live-preview in Three.js. Then I use blender and a python script to assemble the whole scene in blender, based on the chosen materials and 3D objects, which are then exported as *.glb.

The material that made the above images looked fine in Three.js before:

PS: And you're right, I tried both BLEND and OPAQUE to see if it makes any difference in the gltf but that doesn't seem to be it. Also, either way the color should be applied correctly.

[elalish]

Thanks, that's helpful. We use three.js under the hood, so what this really implies is that the exporters and importers are not managing to do a lossless round-trip of a three.js material. Not sure if the bug is on the three.js side, the Blender side, or if you've just selected some combination that's not representable in glTF. You can set a break point in MV, maybe down in the three.js render method and take a look at the MeshPhysicalMaterial you actually ended up with. Hopefully that will help with filing bugs on the upstream tools.

[Bersaelor]

. You can set a break point in MV, maybe down in the three.js render method and take a look at the MeshPhysicalMaterial you actually ended up with.

Good idea, I'm not super deep in the tools necessary to put breakpoints into dependencies, but will see what I can do (I.e. the three.js thats in the node_modules folder is minified and not running the src folder, I have to google how to put breakpoints in the un-minified code).

In the mean time, I built the following minimal example:

// npm installed is "three": "^0.146.0",
import { Collada, ColladaLoader } from 'three/examples/jsm/loaders/ColladaLoader';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';

import planga from './planga.glb';

//...
                scene.current = new THREE.Scene();
                scene.current.background = new THREE.Color( 0xffffff );

                const loader = new GLTFLoader( );
                const dracoLoader = new DRACOLoader();
                dracoLoader.setDecoderPath( 'https://www.gstatic.com/draco/versioned/decoders/1.4.1/' );
                loader.setDRACOLoader( dracoLoader );

                loader.load(planga, function (main) {
                    const g = main.scene.childNamed("Glasses")
                    if (scene.current) scene.current.add(g);
                })
// ... plus the usual camera positioning and render mounting

planga is the same model linked above and the result looks fine (you can see the transmission since the internal metal parts of correctly blurred on the left and right end of the lenses:

so it seems plain THREE.js can load the linked model fine.

[Bersaelor]

Alright, I managed to put a breakpoint in
node_modules/@google/model-viewer/src/three-components/Renderer.ts/Renderer/render method and log the scene:

scene._model.children[0].children[0].children[0] should be the part named FRONT.
console.log(scene._model.children[0].children[0].children[0].material):

Details

MeshPhysicalMaterial {isMaterial: true, uuid: 'd8b0f0d6-527e-4f38-b7f1-2c7504fc724f', name: 'Plastic.002', type: 'MeshPhysicalMaterial', blending: 1, …}
_alphaTest: 0
_clearcoat: 0
_iridescence: 0
_listeners: {dispose: Array(1)}
_sheen: 0
_transmission: 1
alphaMap: null
alphaTest: ƒ () {\n      return this._alphaTest;\n    }
alphaToCoverage: false
aoMap: null
aoMapIntensity: 1
attenuationColor: Color {isColor: true, r: 1, g: 1, b: 1}
attenuationDistance: Infinity
blendDst: 205
blendDstAlpha: null
blendEquation: 100
blendEquationAlpha: null
blending: 1
blendSrc: 204
blendSrcAlpha: null
bumpMap: null
bumpScale: 1
clearcoat: ƒ () {\n      return this._clearcoat;\n    }
clearcoatMap: null
clearcoatNormalMap: null
clearcoatNormalScale: Vector2 {x: 1, y: -1}
clearcoatRoughness: 0
clearcoatRoughnessMap: null
clipIntersection: false
clippingPlanes: null
clipShadows: false
color: Color {isColor: true, r: 0.1119324266910553, g: 0.061246052384376526, b: 0.6307571530342102}
colorWrite: true
defines: {STANDARD: '', PHYSICAL: ''}
depthFunc: 3
depthTest: true
depthWrite: true
displacementBias: 0
displacementMap: null
displacementScale: 1
dithering: false
emissive: Color {isColor: true, r: 0, g: 0, b: 0}
emissiveIntensity: 1
emissiveMap: null
envMap: null
envMapIntensity: 1
flatShading: false
fog: true
ior: 1.5
iridescence: ƒ () {\n      return this._iridescence;\n    }
iridescenceIOR: 1.3
iridescenceMap: null
iridescenceThicknessMap: null
iridescenceThicknessRange: (2) [100, 400]
isMaterial: true
isMeshPhysicalMaterial: true
isMeshStandardMaterial: true
lightMap: null
lightMapIntensity: 1
map: null
metalness: 0
metalnessMap: null
name: 'Plastic.002'
normalMap: Texture {isTexture: true, uuid: 'bc0eec5b-3209-48e1-a84e-c5ae967fc084', name: 'plastic-ozean-normal', source: Source, mipmaps: Array(0), …}
normalMapType: 0
normalScale: Vector2 {x: 1, y: -1}
opacity: 1
polygonOffset: false
polygonOffsetFactor: 0
polygonOffsetUnits: 0
precision: null
premultipliedAlpha: false
reflectivity: ƒ () {\n        return clamp(2.5 * (this.ior - 1) / (this.ior + 1), 0, 1);\n      }
roughness: 0.4300000071525574
roughnessMap: null
shadowSide: 0
sheen: ƒ () {\n      return this._sheen;\n    }
sheenColor: Color {isColor: true, r: 0, g: 0, b: 0}
sheenColorMap: null
sheenRoughness: 1
sheenRoughnessMap: null
side: 2
specularColor: Color {isColor: true, r: 1, g: 1, b: 1}
specularColorMap: null
specularIntensity: 1
specularIntensityMap: null
stencilFail: 7680
stencilFunc: 519
stencilFuncMask: 255
stencilRef: 0
stencilWrite: false
stencilWriteMask: 255
stencilZFail: 7680
stencilZPass: 7680
thickness: 0
thicknessMap: null
toneMapped: true
transmission: ƒ () {\n      return this._transmission;\n    }
transmissionMap: null
transparent: false
type: 'MeshPhysicalMaterial'
userData: {associations: {…}}
uuid: 'd8b0f0d6-527e-4f38-b7f1-2c7504fc724f'
version: 1
vertexColors: false
visible: true
wireframe: false
wireframeLinecap: 'round'
wireframeLinejoin: 'round'
wireframeLinewidth: 1
id: 67

The line

color: Color {isColor: true, r: 0.1119324266910553, g: 0.061246052384376526, b: 0.6307571530342102}

looks like it should be fine to me, still no color in model-viewer.

In comparison let's also log the MeshPhysicalMaterial from the minimal Three.js example above, that just loads the glb and shows it fine (JSON.stringified this time):

Details

{
  "uuid": "726f8e51-40a3-4c9b-903b-ec425b2c4ff7",
  "type": "MeshPhysicalMaterial",
  "name": "Plastic.002",
  "color": 1839008,
  "roughness": 0.4300000071525574,
  "metalness": 0,
  "sheen": 0,
  "sheenColor": 0,
  "sheenRoughness": 1,
  "emissive": 0,
  "specularIntensity": 1,
  "specularColor": 16777215,
  "clearcoat": 0,
  "clearcoatRoughness": 0,
  "iridescence": 0,
  "iridescenceIOR": 1.3,
  "iridescenceThicknessRange": [
    100,
    400
  ],
  "normalMap": "f7102614-c76b-4efa-9a3d-922399780694",
  "normalMapType": 0,
  "normalScale": [
    1,
    -1
  ],
  "envMapIntensity": 1,
  "reflectivity": 0.5,
  "transmission": 1,
  "thickness": 0,
  "attenuationColor": 16777215,
  "side": 2,
  "depthFunc": 3,
  "depthTest": true,
  "depthWrite": true,
  "colorWrite": true,
  "stencilWrite": false,
  "stencilWriteMask": 255,
  "stencilFunc": 519,
  "stencilRef": 0,
  "stencilFuncMask": 255,
  "stencilFail": 7680,
  "stencilZFail": 7680,
  "stencilZPass": 7680,
  "textures": [
    {
      "uuid": "f7102614-c76b-4efa-9a3d-922399780694",
      "name": "plastic-ozean-normal",
      "image": "ae59df60-74e5-454e-87d5-0faa853eafa7",
      "mapping": 300,
      "repeat": [
        1,
        1
      ],
      "offset": [
        0,
        0
      ],
      "center": [
        0,
        0
      ],
      "rotation": 0,
      "wrap": [
        1000,
        1000
      ],
      "format": 1023,
      "type": 1009,
      "encoding": 3000,
      "minFilter": 1008,
      "magFilter": 1006,
      "anisotropy": 1,
      "flipY": false,
      "premultiplyAlpha": false,
      "unpackAlignment": 4,
      "userData": {
        "mimeType": "image/jpeg"
      }
    }
  ],
  "images": [
    
  ]
}

[elalish]

Yeah, this is strange, I too looked at the materials between the working and non-working version and couldn't find any difference. Can you repro with a simpler model? Perhaps with just one material?

[mrdoob]

Where can I find a link to the working version of the model?

[Bersaelor]

Where can I find a link to the working version of the model?

It's the same model one time I imported it via a plain THREE.js implementation (works, see #4047 (comment) ) .
But then importing via modelviewer the material failed to work (as in og post)

[elalish]

I just had an inspiration on why this is different in MV vs the three.js editor: MV renders transparently, compositing afterward into whatever DOM content is behind it, while three's editor renders a background into its own buffers. You can see the difference in our own editor by loading up your model and in the second tab, choosing a different environment image and then checking "use environment as skybox" - suddenly it looks like the other renderers.

This is a helpful feature of MV for web design, but I believe there's also improvement that could be made in three's shaders for transparent materials with no background images behind them. As a workaround, you can set a simple single-color skybox-image and use environment-image="neutral" or whatever you want.

[mrdoob]

So if there's no background we render the environment as background in the transmission buffer?

[elalish]

So if there's no background we render the environment as background in the transmission buffer?

I hope the answer to this is no. I just want to see if our little non-PBR hack for rendering to a transparent buffer can be improved to save the transmissive color better.

[mrdoob]

Hmm, I can't see how to make that work...

If it was me, I would make it so <model-viewer>'s background (white by default?) goes opaque and logs a warning in the console saying that models with refractive materials only work when <model-viewer> has a opaque background and display the API to change the background color.

And then remove the hack from three.js shader 😅

[elalish]

I think I have a promising solution: mrdoob/three.js#25819

[elalish]

Fixed by #4314 - by pulling in the above change.