Support vertical layering of opaque BSDFs

[jstone-lucasfilm]

This changelist extends vertical layering to opaque BSDFs, adopting the convention that every BSDF in the PBS library defines a vertical-layering transmittance: the fraction of incident light that passes through the BSDF to the layers beneath it. Interface BSDFs such as dielectric_bsdf and sheen_bsdf transmit all of the energy they do not reflect, while opaque BSDFs such as oren_nayar_diffuse_bsdf and conductor_bsdf treat their weight input as a statistical coverage of the surface, transmitting light only through the uncovered fraction.

Previously, an opaque BSDF used as the top input of a layer node fully occluded its base regardless of weight, so even a zero-weight lobe occluded the layers beneath it. With this change, layering an opaque BSDF over a base is equivalent to mixing the two BSDFs by the coverage, making a zero-weight lobe transparent to its base and preserving energy conservation throughout.

Specific changes:

• Update the throughput of the opaque BSDF closures in hardware shading languages from full occlusion to the uncovered fraction 1 - weight.
• Extend MDL shader generation to support vertical layering of opaque BSDFs, passing the layer base into their existing weighted_layer composition.
• Add a Vertical-Layering Transmittance section to the PBR specification, defining the transmittance of each BSDF in the PBS library, its composition by the mix, layer, add, and multiply nodes, and the energy conservation invariant relating transmittance to directional albedo.
• Add test graphs validating that layering an opaque BSDF over a base matches the equivalent mix (pixel-identical in GLSL), and that a zero-weight opaque top leaves its base unoccluded.

Notes for reviewers:

• No shading models in the standard libraries change appearance: all layer tops in libraries/bxdf are interface BSDFs or unit-weight opaque BSDFs. User materials that layer a partial-weight opaque BSDF will brighten, as the base now correctly receives the uncovered fraction of light.
• Shader-semantic multiply nodes intentionally preserve transmittance (attenuating only the scattered response), so multiply-scaled opaque top layers (e.g. the topMix of LamaLayer) still fully occlude their base, while mix-scaled top layers interpolate transmittance instead.
• OSL layer closure semantics are implemented by host renderers; the new specification section defines the expected behavior for those implementations.

[jstone-lucasfilm]

In addition to the list of reviewers above, I'm CC'ing @krohmerNV and @jreichel-nvidia for their thoughts from the MDL perspective.

[tdavidovicNV]

I really like this, definitely good direction.

I have two concerns about the MDL, which just reinforce the call for my more MDL capable colleagues to take a look. I tried to point out the code that looks worrying, but I don't know enough about MDL to propose actually working fixes.

[tdavidovicNV]

The new SSS layering path forwards mxp_base.surface.scattering, but the returned volume is still constructed exclusively from the top SSS node.

Shouldn’t this be something along the lines of:

coverage = saturate(mxp_weight * mxp_top_weight)

result.surface =
    weighted_layer(coverage, top.surface, base.surface)

result.volume =
    volume_mix(coverage, top.volume,
               1 - coverage, base.volume)

result.ior = base.ior

In particular, layer(subsurface(weight=0), baseSubsurface) should preserve the base volume exactly. I will leave the exact code to more MDL-capable people.

[tdavidovicNV]

The conductor now forwards mxp_base.volume, but the returned material still uses the conductor’s IOR unconditionally no matter the top weight:

ior: mxp_ior

The important point is that layer(conductor(weight=0), base) should preserve the base IOR, I think.

I don't think the conductor's IOR should affect the volume's IOR (at least that's my reading of this code), unless we go full IOR accumulation route.

[tdavidovicNV]

After looking into this more, I need to walk back my earlier agreement with this PR. I agree with most of the change, but not with the proposed behavior of multiply.

The proposed spec currently says two things:

1. For an opaque BSDF, its weight controls coverage, with the remaining throughput to lower layers given by (T=1-w). (spec text)
2. A multiply scales the BSDF response but leaves its throughput (T) unchanged. (spec text)

This gives these two graphs different meanings:

oren_nayar_diffuse_bsdf(weight=0.5)
multiply(oren_nayar_diffuse_bsdf(weight=1), 0.5)

The first one covers half of the surface and lets half of the base show through. The second one still covers the whole surface. It only makes the Oren–Nayar response darker, with the removed light treated as absorption.

At zero, the difference is even clearer:

oren_nayar_diffuse_bsdf(weight=0)

means no top layer, while:

multiply(oren_nayar_diffuse_bsdf(weight=1), 0)

means a completely black layer which still hides the base.

I don’t think these should be different. This is also not what I intended in #2971, where multiply(top, 0) was meant to be empty and therefore leave the base unchanged. That PR only discussed the zero case, so it did not define the behavior of partial multipliers.

There is also a practical cross-target problem. Current genosl writes both a node’s weight and an external multiply as scalar multiplication of an OSL closure. OSL’s reference layer implementation uses that scalar when deciding how much of the base is visible. It therefore treats both cases as coverage.

OSL could probably be extended to support the distinction proposed here, but this would require a change to the closure contract and corresponding changes in OSL renderers. It is not just a MaterialX codegen change.

MDL can represent the distinction because it has separate operations for layer weight and BSDF tinting. However, MaterialX’s MDL layer codegen does not currently handle a multiply node as the layer top, and this PR does not add or test that case.

I would prefer that multiply(bsdf, s) scales the effective weight of the BSDF by (s). In other words, multiplying a fully weighted BSDF by 0.5 should be equivalent to setting its weight to 0.5.

For an opaque BSDF, this means reducing its coverage. Multiplying by 0.5 leaves half the surface uncovered, while multiplying by zero removes the layer completely.

Since the spec defines throughput as (T=1-w) for opaque BSDFs, scaling the effective weight by (s) gives:

$$ T_{\mathrm{out}}=1-s(1-T_{\mathrm{in}}) $$

Given GLSL and OSL are our reference renders, and this proposal introduces something OSL currently cannot easily express, I am worried.

[tdavidovicNV]

(I am gonna walk back the recommendation, because LamaLayer and LamaConductor utilize the multiply in conflicting ways, the first one reducing coverage while the second one keeping coverage and just changing tint.. so the story is way more complicated with compatibility in mind)

[tdavidovicNV]

After looking into this much more, I need to walk back both my earlier agreement with the multiply part of this PR and my first suggestion for fixing it. The current behavior is less consistent than I expected, and I don't think either meaning of multiply is ready to standardize yet.

The underlying problem is that scaling a BSDF response and scaling its coverage are different operations. At zero, this is the difference between an absent BSDF, which reveals the layer below, and a black absorbing BSDF, which still hides it.

#3022 currently makes that distinction by saying that an elemental BSDF's own weight controls coverage, while multiply scales only the response and leaves vertical-layer transmittance unchanged.

I tested the same material with a neutral, reflection-only dielectric top in several backends:

layer(top(weight=s), base)
layer(multiply(top, s), base)

The results are not consistent (the base is green, the top is dielectric):

Backend	top.weight=s	multiply(top,s)
GLSL	Top response decreases and base pass-through increases	Top response is scaled, but attenuation of the base is unchanged
OSL	Top response decreases and base pass-through increases	Same result as weight=s
Arnold using generated OSL	Same as OSL	Same as OSL
Karma CPU	Top response decreases and base pass-through increases	The complete layered result is scaled
MDL code generation	A direct elemental top maps to a weighted layer	A multiply top does not carry the outer base

This behavior is not inferred from the renders alone. GLSL includes the dielectric weight in the directional albedo used to compute pass-through, and OSL uses the weighted closure when deciding how much of the base remains. In the vertical-layering calculation, both therefore treat a reduced dielectric weight as reduced effective coverage. The renders confirm that behavior. They also show that multiply has no shared behavior today. Arnold confirms the generated OSL behavior, but is not an independent lowering.

OSL has no way to scale only the response of an arbitrary closure while leaving its layering weight unchanged. Closure multiplication changes the closure weight, which is also used for coverage. Pushing the multiplier into the BSDF's color or tint parameters is not an option either: the input may be an arbitrary composition, and the language has no operation that can inspect that composition and modify its parameters.

Mixing with opaque black does not solve this. Consider:

multiply(white_bsdf(weight=0.5), 0.5)

Response-only scaling would produce a half-covered grey BSDF. OSL closure multiplication produces a quarter-covered white BSDF. Mixing the half-covered white BSDF with opaque black produces half black, one quarter white, and one quarter uncovered: 75% coverage.

All three look the same over black, but behave differently when layered. So the proposed response-only meaning of a generic BSDF multiply cannot be expressed using the current OSL closure operations. It would require a new closure operation and renderer support.

MDL can represent response scaling with df::tint, but the current MaterialX layer lowering cannot carry the outer base through an arbitrary multiply, mix, or add top.

My #3017 proposal did not settle multiply either. It introduced pass-through as something separate from directional albedo, and its implementation notes were compatible with closure weight affecting pass-through, but that was not enough to define the semantics of the multiply node.

There are also two separate questions involving mix.

An ordinary mixed top:

layer(mix(fg=A, bg=B, mix=w), base)

needs a pass-through value of its own. glTF and OpenPBR already use this form, and the linear rule proposed in #3022 is the natural one:

$$ T_{\mathrm{mix}} = (1-w)T_B + wT_A $$

I think this should remain part of the 1.39.6 definition.

The separate question is how to reduce the coverage of an arbitrary compound top. Under the proposed rules this can be written as:

layer(mix(bg=zero, fg=compound, mix=s), base)

Here zero means an absent BSDF, not an opaque black BSDF.

GLSL and OSL give the expected coverage behavior for this graph. In my tests Karma produces black at s=0, and the current MDL lowering loses the outer base. This does not make the ordinary mix rule ambiguous, but it does show that compound-top coverage is not portable today.

Lama also shows why this is not merely theoretical. LamaLayer.topMix applies to an arbitrary top material and is currently implemented with multiply(materialTop, topMix), while other Lama nodes use multiply to tint a response. The same MaterialX operation is already serving two different purposes.

Other material systems generally give these operations separate names. MDL has df::tint for response scaling and df::weighted_layer for presence, while Manuka and OpenPBR expose explicit material mixing and weighted layering. This does not establish either meaning as the natural behavior of a generic BSDF multiply.

So, for 1.39.6, I would suggest the following scope:

• Define vertical-layer pass-through separately from directional albedo and path throughput.
• Define it for the existing interface BSDFs, elemental opaque BSDFs, ordinary mix results, and nested layer results.
• Extend the existing vertical_layering.mtlx cases to cover opaque weights 0, 0.5 and 1, and actually compare layer(top(weight=s), base) with mix(bg=base, fg=top(weight=1), mix=s) within both GLSL and OSL.
• Add a normal mix-as-top test, since the standard glTF and OpenPBR graphs depend on it.
• Add MDL generator and compile coverage for the elemental opaque cases changed by this PR. The same MaterialX documents can be used as manual downstream tests in Karma.

The effect of multiply on layer pass-through still needs a separate decision. The same is true for add, the meaning of BSDF weight when the BSDF is standalone or at the bottom of a stack, and a portable way to control the coverage of an arbitrary compound top.

I think those questions should become focused follow-up issues. One should cover the relationship between response, coverage, weight, and multiply, including the difference between an absent BSDF and an absorbing black BSDF. Another should cover compound-top coverage, including Lama's topMix and whether MaterialX eventually needs an explicit weighted-layer operation. The renderer matrix and the test materials above can be used as starting cases for both.

That would still give #3022 a useful result for 1.39.6: opaque BSDFs become usable as top layers, while the unresolved multiply and compound-coverage questions get a proper investigation instead of being decided indirectly by the first formal definition.