[Native] Honor depthCullingState.depthTest on the native engine#18558
Conversation
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
There was a problem hiding this comment.
Pull request overview
This PR fixes a Babylon Native rendering gap where toggling engine.depthCullingState.depthTest (notably by EffectRenderer / EffectWrapper fullscreen passes) was not reflected in the native command stream, causing depth-test state to “leak” from previous draws and potentially discard fullscreen fragments.
Changes:
- Add native-side tracking for whether depth testing is currently encoded/enabled, and reconcile it before each draw.
- Update
setDepthBuffer()to keep the shared_depthCullingState.depthTestin sync and encode the corresponding native command. - Flush depth-test enable/disable state at the start of
drawElementsType/drawArraysTypeso directdepthCullingState.depthTestmutations are honored.
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18558/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18558/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18558/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
WebGL2 visualization test reporter: |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
|
Visualization tests for WebGPU |
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
bghgary
left a comment
There was a problem hiding this comment.
[Reviewed by Copilot on behalf of @bghgary]
Targeted fix for one slice of #1106 (native engine not honoring state set through the property objects) — worth linking there. The open setDepthFunction thread is a real gap in the new _flushDepthTestState tracking; should be closed or consciously deferred like the stencil case.
This EffectRenderer fullscreen-pass test rendered all-black on Native because the native engine did not honor depth-test toggles made through engine.depthCullingState.depthTest (used by EffectRenderer). Fixed upstream in the native engine (BabylonJS/Babylon.js#18558); re-enable the test here. Depends on a babylonjs dependency bump that includes the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
EffectRenderer (and ThinDepthPeelingRenderer) disable depth testing by setting engine.depthCullingState.depthTest directly. On WebGL/WebGPU this is flushed by applyStates() at draw time, but the native engine only honored depth state set through the explicit setDepthBuffer() command and never flushed depthCullingState before a draw, so the toggle was silently dropped. A fullscreen EffectWrapper pass then kept the previous draw's depth test and every fragment was discarded, producing an all-black target. Reconcile depthCullingState.depthTest before each native draw to match the cross-engine contract: setDepthBuffer() keeps the state object authoritative and _flushDepthTestState() emits the command when the direct toggle differs from what was last encoded. No public API change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
59d94a6 to
889cfb1
Compare
Route setDepthFunction() through _encodeDepthTest() so the tracked _depthTestEnabled state always matches the encoded COMMAND_SETDEPTHTEST. Previously setDepthFunction encoded the compare function directly, which (since DEPTH_TEST_ALWAYS doubles as "disabled") could silently re-enable depth testing while it was logically disabled, leaving _flushDepthTestState unable to reconcile the desync before the next draw. Also initialize _stencilTest to false (it was a no-op statement), so getStencilBuffer() returns a proper boolean before setStencilBuffer runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks @bghgary — addressed both points in 7d90b87:
This PR is a targeted slice of BabylonJS/BabylonNative#1106 (Handle Engine States in Babylon Native) — linked there as well. |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18558/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18558/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18558/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
WebGL2 visualization test reporter: |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
cc @CedricGuillemet or @bghgary for review please :-) |
This EffectRenderer fullscreen-pass test rendered all-black on Native because the native engine did not honor depth-test toggles made through engine.depthCullingState.depthTest (used by EffectRenderer). Fixed upstream in the native engine (BabylonJS/Babylon.js#18558); re-enable the test here. Depends on a babylonjs dependency bump that includes the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This EffectRenderer fullscreen-pass test rendered all-black on Native because the native engine did not honor depth-test toggles made through engine.depthCullingState.depthTest (used by EffectRenderer). Fixed upstream in the native engine (BabylonJS/Babylon.js#18558); re-enable the test here. Depends on a babylonjs dependency bump that includes the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This EffectRenderer fullscreen-pass test rendered all-black on Native because the native engine did not honor depth-test toggles made through engine.depthCullingState.depthTest (used by EffectRenderer). Fixed upstream in the native engine (BabylonJS/Babylon.js#18558); re-enable the test here. Depends on a babylonjs dependency bump that includes the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
[Native] Honor
depthCullingState.depthTeston the native engineProblem
EffectRenderer(used byEffectWrapper-based fullscreen passes) disables depthtesting by mutating the shared depth-culling state directly:
On the WebGL/WebGPU engines this is flushed to the GPU by
applyStates(), which iscalled from
drawElementsType/drawArraysTypeand readsdepthCullingState.The native engine does not go through that
applyStates()path — its draw methodsencode a draw command directly, and depth state is only ever changed through the
explicit
setDepthBuffer()/setDepthFunction()commands. As a result, a depth-testtoggle done through
engine.depthCullingState.depthTestnever reaches the nativecommand stream.
The visible effect: a fullscreen
EffectWrapperquad keeps the depth-test state of thepreviously rendered geometry (e.g.
LEQUAL). Drawn at clip-spacez = 0.5against arender target whose depth buffer is not at the far value, every fragment fails the depth
test and is discarded, so the pass produces an all-black target even though the pixel
shader output is correct.
ThinDepthPeelingRenderertogglesdepthCullingState.depthTestdirectly as well and isaffected by the same gap.
Fix
Make the native engine honor
depthCullingState.depthTestthe same way the otherengines do:
setDepthBuffer()keeps_depthCullingState.depthTestin sync (matching the baseAbstractEngine.setDepthBuffer, which is literallythis._depthCullingState.depthTest = enable)._flushDepthTestState()reconciles the encoded depth-testenable with
depthCullingState.depthTestand emits the command only when it changed.This keeps the existing explicit
setDepthBuffer()/setDepthFunction()paths workingunchanged (they already set the state object / are the source of truth) while also
honoring direct
depthCullingState.depthTestmutations.No public API change.
Testing
Validated on Babylon Native (Win32 / D3D11) with the Playground validation suite:
EffectRendereronion-skin/blur fullscreen-pass scene renderscorrectly instead of all black.
passes, including depth- and transparency-sensitive scenes).
Notes
EffectRendereralso togglesstencilState.stencilTestdirectly. That has the samelatent gap on native but defaults to
falseand has no reproducing scene today, so it isintentionally left out of this change to keep it minimal; it can be addressed with the
same pattern if a repro appears.
Partial fix for BabylonJS/BabylonNative#1106 (Handle Engine States in Babylon Native).