Direct lighting diffuse/specular energy preservation and IBL consistency

[zyz23333]

Direct lighting diffuse/specular energy preservation and IBL consistency

Summary

Filament's BRDF documentation already identifies two related energy issues:

• Energy gain in diffuse reflectance
• Energy loss in specular reflectance

The second issue, specular energy loss, is already addressed in the standard shading model through the multiscattering specular energy compensation term:

pixel.energyCompensation = 1.0 + pixel.f0 * (1.0 / pixel.dfg.y - 1.0);
Fr *= pixel.energyCompensation;

This scales the single-scattering GGX specular lobe to compensate for energy lost by not explicitly simulating multiple microfacet scattering.

However, the first issue, energy gain in diffuse reflectance, is still unresolved for direct lighting. The standard shading model currently evaluates diffuse and specular independently and adds them:

vec3 Fr = specularLobe(pixel, light, h, NoV, NoL, NoH, LoH);
vec3 Fd = diffuseLobe(pixel, NoV, NoL, LoH);

// TODO: attenuate the diffuse lobe to avoid energy gain

vec3 color = Fd + Fr * pixel.energyCompensation;

The problem is that the specular lobe is compensated for multiscattering, but the diffuse lobe is still evaluated as if the full incident light reached the diffuse scattering event. In Filament's own terminology, the specular-side energy loss is compensated, but the diffuse-side energy gain is not removed.

This issue proposes exposing the effective specular directional albedo after multiscattering compensation:

vec3 compensatedSpecularDirectionalAlbedo(const PixelParams pixel) {
    return saturate(specularDFG(pixel) * pixel.energyCompensation);
}

and using it to attenuate the diffuse lobe:

Fd *= 1.0 - compensatedSpecularDirectionalAlbedo(pixel);

This completes the diffuse-side counterpart of the existing specular energy compensation and makes direct lighting, prefiltered IBL, and importance-sampled IBL use the same energy accounting.

---

Background: compensation vs preservation

Filament's documentation describes two different problems:

1. Energy gain in diffuse reflectance

The documentation states that the Lambert diffuse BRDF does not account for light that reflects at the surface and therefore cannot participate in the diffuse scattering event.

In a layered interpretation of the standard material, incident light first interacts with the specular interface. Only the part not reflected by the specular layer should reach the diffuse substrate.

Conceptually:

incoming energy
    = specular reflected energy
    + energy transmitted to diffuse scattering

Therefore, if the specular layer reflects an amount E, the diffuse lobe should receive approximately:

1 - E

of the incident energy.

2. Energy loss in specular reflectance

The Cook-Torrance GGX BRDF used by Filament is a single-scattering microfacet model. At high roughness, light can bounce between microfacets before escaping. A single-scattering model discards these paths through masking and shadowing, causing rough specular materials, especially metals, to lose energy.

Filament already compensates this by scaling the single-scattering specular lobe:

pixel.energyCompensation = 1.0 + pixel.f0 * (1.0 / pixel.dfg.y - 1.0);

This can be written as:

$$ W(v, \alpha, f_0) = 1 + f_0 \left( \frac{1}{r(v,\alpha)} - 1 \right) $$

where:

$$ r(v,\alpha) = \int_{\Omega} D(l,v,\alpha) V(l,v,\alpha) \langle n \cdot l \rangle dl $$

In Filament's multiscattering DFG LUT, this term is stored in:

pixel.dfg.y

so:

$$ r(v,\alpha) = pixel.dfg.y $$

The current implementation applies:

$$ f_r(l,v) = W(v,\alpha,f_0) f_{ss}(l,v) $$

where f_ss is the original single-scattering specular lobe.

This part is correct and matches the documentation: it compensates the specular energy loss.

The missing part is that once the specular lobe has been compensated, the diffuse lobe should be reduced by the energy reflected by the compensated specular layer.

---

Directional albedo terms

To avoid ambiguity, this issue uses the following terms:

W: specular energy compensation weight

vec3 W = pixel.energyCompensation;

This is the multiplier applied to the single-scattering specular BRDF.

It is not itself the reflected specular energy. It is a lobe scale factor.

E_ss: single-scattering specular directional albedo

The directional albedo of a BRDF is the fraction of unit incident radiance reflected by that BRDF for a given view direction:

$$ E(v) = \int_{\Omega} f(l,v) \langle n \cdot l \rangle dl $$

For the single-scattering specular lobe with Schlick Fresnel, Filament already reconstructs this from the DFG LUT:

vec3 specularDFG(const PixelParams pixel) {
#if defined(SHADING_MODEL_CLOTH)
    return pixel.f0 * pixel.dfg.z;
#else
#if defined(MATERIAL_HAS_SPECULAR_COLOR_FACTOR) || defined(MATERIAL_HAS_SPECULAR_FACTOR)
    return mix(pixel.dfg.xxx, pixel.dfg.yyy, pixel.f0) * pixel.specular;
#else
    return mix(pixel.dfg.xxx, pixel.dfg.yyy, pixel.f0);
#endif
#endif
}

For the standard shading model, this corresponds to:

$$ E_{ss}(v,\alpha,f_0) = (1 - f_0) DFG_1(v,\alpha) + f_0 DFG_2(v,\alpha) $$

or in shader terms:

vec3 E_ss = specularDFG(pixel);

In Filament's multiscattering DFG pre-integration:

dfg.x = DFG_1 = Fresnel Fc-weighted term
dfg.y = DFG_2 = non-Fresnel term, also used as r

and the reconstruction is:

mix(dfg.xxx, dfg.yyy, f0)

which is:

$$ (1 - f_0) DFG_1 + f_0 DFG_2 $$

Where specularDFG(pixel) comes from

specularDFG(pixel) is Filament's reconstruction of the Fresnel-weighted specular DFG term used by the split-sum approximation for specular image-based lighting.

The specular IBL integral is:

$$ L_{spec}(v) = \int_{\Omega} L_i(l) f_{spec}(l,v) \langle n \cdot l \rangle dl $$

For the GGX Cook-Torrance specular BRDF, this is approximately:

$$ L_{spec}(v) = \int_{\Omega} L_i(l) D(l,v,\alpha) V(l,v,\alpha) F(v,h,f_0) \langle n \cdot l \rangle dl $$

The split-sum approximation separates the filtered lighting term from the BRDF integration term:

$$ L_{spec}(v) \simeq LD(n,\alpha) \cdot DFG(\langle n \cdot v \rangle,\alpha,f_0) $$

where LD is the prefiltered environment radiance and DFG is the pre-integrated BRDF factor. In a white furnace environment, LD is approximately one, so the DFG term itself represents the specular directional albedo of the lobe.

With Schlick Fresnel:

$$ F(v,h) = f_0 + (f_{90} - f_0) F_c(\langle v \cdot h \rangle) $$

and:

$$ F_c(\langle v \cdot h \rangle) = (1 - \langle v \cdot h \rangle)^5 $$

Filament's multiscattering DFG reconstruction assumes:

$$ f_{90} = 1 $$

so:

$$ F(v,h) = f_0 + (1 - f_0)F_c $$

or equivalently:

$$ F(v,h) = f_0 \cdot 1 + (1 - f_0) \cdot F_c $$

Substituting this into the DFG integral gives:

$$ DFG(\langle n \cdot v \rangle,\alpha,f_0) = \int_{\Omega} D V \left[f_0 + (1 - f_0)F_c\right] \langle n \cdot l \rangle dl $$

After separating the terms that depend on f0:

$$ DFG(\langle n \cdot v \rangle,\alpha,f_0) = f_0 \int_{\Omega} D V \langle n \cdot l \rangle dl + (1 - f_0) \int_{\Omega} D V F_c \langle n \cdot l \rangle dl $$

Filament stores these two terms in the multiscattering DFG LUT as:

dfg.x = DFG_1 = integral(D * V * Fc * NoL)
dfg.y = DFG_2 = integral(D * V      * NoL)

The DFG pre-integration code follows this layout:

float Fc = pow(1 - VoH, 5.0f);
r.x += Gv * Fc;
r.y += Gv;

The shader reconstruction is therefore:

mix(dfg.xxx, dfg.yyy, f0)

which expands to:

$$ (1 - f_0)DFG_1 + f_0DFG_2 $$

This is exactly what specularDFG(pixel) returns for the standard shading model:

vec3 E_ss = specularDFG(pixel);

This value is the Fresnel-weighted directional albedo of the single-scattering specular lobe. It is not yet the directional albedo of the lobe that is finally used for lighting, because direct lighting and IBL later multiply the specular lobe by:

pixel.energyCompensation

Therefore:

specularDFG(pixel)

corresponds to:

$$ E_{ss} $$

while:

saturate(specularDFG(pixel) * pixel.energyCompensation)

corresponds to:

$$ E_{comp} $$

This distinction is important: specularDFG(pixel) tells us how much energy the uncompensated single-scattering lobe reflects, while E_comp tells us how much energy the compensated lobe used by the renderer reflects.

E_comp: compensated specular directional albedo

The specular BRDF actually used for lighting is not f_ss; it is:

$$ f_r(l,v) = W(v,\alpha,f_0) f_{ss}(l,v) $$

Therefore, the effective directional albedo of the specular lobe used by the renderer is approximately:

$$ E_{comp}(v,\alpha,f_0) = \mathrm{saturate} \left( W(v,\alpha,f_0) E_{ss}(v,\alpha,f_0) \right) $$

or:

vec3 E_comp = saturate(specularDFG(pixel) * pixel.energyCompensation);

This is the amount of energy reflected by the compensated specular layer. The diffuse lobe should receive the remaining energy:

$$ 1 - E_{comp} $$

The saturate is important because the DFG reconstruction, multiscattering approximation, colored specular factors, and roughness remapping are approximate. Clamping prevents negative diffuse energy in edge cases.

---

Direct lighting

The current standard direct lighting path is effectively:

$$ L_{direct} \propto F_d + W F_{ss} $$

or in code:

vec3 Fr = specularLobe(pixel, light, h, NoV, NoL, NoH, LoH);
vec3 Fd = diffuseLobe(pixel, NoV, NoL, LoH);

vec3 color = Fd + Fr * pixel.energyCompensation;

This compensates the specular lobe but still lets the diffuse lobe use the full incident energy. The result can over-preserve energy, especially for rough dielectric materials under strong direct lighting or grazing views.

The proposed direct lighting model is:

$$ L_{direct} \propto (1 - E_{comp}) F_d + W F_{ss} $$

where:

$$ E_{comp} = \mathrm{saturate} \left( specularDFG(pixel) \cdot pixel.energyCompensation \right) $$

In shader form:

vec3 compensatedSpecularDirectionalAlbedo(const PixelParams pixel) {
    return saturate(specularDFG(pixel) * pixel.energyCompensation);
}

vec3 surfaceShading(const PixelParams pixel, const Light light, ...) {
    vec3 Fr = specularLobe(pixel, light, h, NoV, NoL, NoH, LoH);
    vec3 Fd = diffuseLobe(pixel, NoV, NoL, LoH);

    vec3 E = compensatedSpecularDirectionalAlbedo(pixel);
    Fd *= 1.0 - E;

    return Fd + Fr * pixel.energyCompensation;
}

This resolves the existing TODO:

// TODO: attenuate the diffuse lobe to avoid energy gain

The important distinction is:

pixel.energyCompensation scales the specular lobe.
compensatedSpecularDirectionalAlbedo estimates how much energy that scaled lobe reflects.
1 - compensatedSpecularDirectionalAlbedo is the energy left for diffuse.

---

IBL consistency

The prefiltered cubemap IBL path already attenuates diffuse by a specular DFG term:

vec3 E = specularDFG(pixel);

Fr = E * prefilteredRadiance(...);
Fr *= pixel.energyCompensation;

Fd = pixel.diffuseColor * diffuseIrradiance * (1.0 - E) * diffuseBRDF;

However, this uses two different energy values:

specular IBL contribution uses E * pixel.energyCompensation
diffuse IBL attenuation uses 1 - E

For consistency, diffuse attenuation should use the same effective specular directional albedo as the specular lobe:

vec3 E = specularDFG(pixel);
vec3 E_comp = saturate(E * pixel.energyCompensation);

Fr = E * prefilteredRadiance(...);
Fr *= pixel.energyCompensation;

Fd = pixel.diffuseColor * diffuseIrradiance * (1.0 - E_comp) * diffuseBRDF;

Equivalently:

$$ L_{specular}^{IBL} \propto E_{comp} $$

and:

$$ L_{diffuse}^{IBL} \propto 1 - E_{comp} $$

This makes the IBL path use the same energy accounting as direct lighting.

---

Importance-sampled IBL

The importance-sampling IBL path currently has:

vec3 E = vec3(0.0); // TODO: fix for importance sampling

Even when the specular radiance is evaluated through importance sampling rather than a prefiltered cubemap, the DFG terms are still available through pixel.dfg.

Therefore, the diffuse attenuation can still use:

vec3 E = compensatedSpecularDirectionalAlbedo(pixel);

instead of leaving E = 0.

This keeps the energy accounting consistent across:

direct lighting
prefiltered cubemap IBL
importance-sampled IBL

---

White furnace intuition

Under a white furnace environment, ignoring AO, clear coat, sheen, transmission, subsurface, and other secondary effects, an energy-preserving dielectric material should not return more than the incident unit radiance.

Without diffuse attenuation, the simplified accounting is:

$$ L \approx 1 + E_{comp} $$

because the diffuse lobe still receives full energy while the specular lobe also reflects its compensated energy.

With the proposed attenuation:

$$ L \approx (1 - E_{comp}) + E_{comp} = 1 $$

This is the expected behavior: the compensated specular layer reflects E_comp, and only the remaining energy reaches the diffuse scattering event.

---

RGB vs scalar attenuation

The smallest change consistent with Filament's current IBL implementation is vector attenuation:

Fd *= 1.0 - E_comp;

This matches the existing IBL pattern:

Fd *= 1.0 - E;

and preserves channel-wise energy accounting for colored specular materials.

A scalar attenuation is also possible:

float e = luminance(E_comp);
Fd *= 1.0 - e;

or:

float e = max3(E_comp);
Fd *= 1.0 - e;

A scalar form can reduce hue shifts for colored specular materials because the diffuse substrate is attenuated uniformly. However, it is a larger behavioral change relative to Filament's current vector IBL attenuation.

This issue proposes the vector form as the smallest internally consistent change, but the choice is worth calling out explicitly.

---

Implementation outline

The implementation can be very small because all required data is already available in PixelParams:

pixel.dfg
pixel.f0
pixel.energyCompensation

No new LUT, material parameter, uniform, or pre-integration pass is required.

One possible implementation is to expose a small helper next to the existing DFG utilities:

vec3 compensatedSpecularDirectionalAlbedo(const PixelParams pixel) {
    return saturate(specularDFG(pixel) * pixel.energyCompensation);
}

Then the standard direct lighting path can replace the existing TODO with:

Fd *= 1.0 - compensatedSpecularDirectionalAlbedo(pixel);

The IBL path can keep using specularDFG(pixel) for specular reconstruction, while using the compensated value for diffuse energy preservation:

vec3 E = specularDFG(pixel);
vec3 E_comp = compensatedSpecularDirectionalAlbedo(pixel);

Fr = E * prefilteredRadiance(...);
Fr *= pixel.energyCompensation;

Fd = pixel.diffuseColor * diffuseIrradiance * (1.0 - E_comp) * diffuseBRDF;

For importance-sampled IBL, the same helper can be used for diffuse attenuation even though the specular radiance itself is sampled:

vec3 E_comp = compensatedSpecularDirectionalAlbedo(pixel);
Fd *= 1.0 - E_comp;

This keeps the change cheap:

one helper
one multiply for direct diffuse attenuation
one multiply for indirect diffuse attenuation
no additional texture fetches beyond the existing DFG lookup
no new precomputed data

The existing pixel.energyCompensation path remains unchanged for the specular lobe. The proposed change only uses that already-computed compensation factor to make the diffuse lobe's energy accounting consistent with the specular lobe that is actually rendered.

---

Related prior art

This proposal is also informed by the energy accounting used in Unreal Engine 5's shading model, which distinguishes conceptually between:

a compensation weight that scales a specular BxDF
an effective directional albedo used to attenuate lower lobes such as diffuse

In the terminology used by this issue, that distinction corresponds to:

W      = specular lobe compensation weight
E_comp = compensated specular directional albedo

The relevant prior-art idea is the same energy-accounting split:

first compensate the specular lobe for single-scattering energy loss,
then use the compensated specular directional albedo to determine how much energy remains for diffuse.

This maps naturally onto Filament's existing concepts:

pixel.energyCompensation  -> W
specularDFG(pixel)        -> E_ss
saturate(E_ss * W)        -> E_comp

The proposed Filament change is expressed entirely in terms of Filament's existing DFG LUT, specularDFG(pixel), and pixel.energyCompensation. It does not require new precomputed data.

---

Engineering tradeoff

This proposal is an approximation, not a full layered BSDF integration.

For direct lighting, the attenuation term:

$$ E_{comp}(v,\alpha,f_0) $$

depends on view direction and roughness through the DFG parameterization. A more complete physical model could depend more explicitly on both light direction and view direction, and would require a more detailed treatment of transmission through the specular microfacet interface into the diffuse substrate.

However, Filament already uses the DFG LUT for:

specular IBL split-sum reconstruction
specular multiscattering energy compensation

Reusing the same DFG terms for diffuse/specular energy preservation has several advantages:

no new LUT
low shader cost
same directional albedo basis as IBL
same compensation term as direct specular lighting
consistent white furnace behavior

---

Expected benefits

This should improve energy preservation for standard materials under:

• white furnace tests;
• high roughness;
• grazing angles;
• strong direct lighting;
• mixed diffuse/specular dielectrics;
• IBL paths where the specular lobe is compensated but diffuse attenuation still uses the uncompensated DFG term.

The intended accounting becomes:

$$ \text{specular reflected energy} \approx E_{comp} $$

$$ \text{diffuse remaining energy} \approx 1 - E_{comp} $$

with:

$$ E_{comp} = \mathrm{saturate} \left( specularDFG(pixel) \cdot pixel.energyCompensation \right) $$

In short, this completes the diffuse-side counterpart of Filament's existing specular multiscattering compensation using DFG data that is already available.