Fix wrapped texture restore after context loss + Native MSAA-RTT support#18469
Conversation
When a texture created via wrapWebGLTexture / wrapNativeTexture / wrapWebGPUTexture is bound as a render-target color attachment, _rebuildRenderTargetWrappers (inside _restoreEngineAfterContextLost) calls createRenderTargetTexture with format=-1 (wrapped textures don't carry one), which throws on Native and silently clobbers the consumer's _hardwareTexture with a fresh RGBA8 texture on WebGL. InternalTexture._rebuild already no-ops for source === Unknown; only the wrapper-side rebuild was inconsistent. - abstractEngine.pure.ts: skip wrapped RTWs (source === Unknown) during rebuild. - engine.pure.ts, thinNativeEngine.pure.ts, webgpuEngine.pure.ts: add updateWrappedWebGLTexture / updateWrappedNativeTexture / updateWrappedWebGPUTexture so consumers can re-supply a fresh handle from their onContextRestoredObservable handler, preserving the InternalTexture identity. - nativeEngine.pure.ts + webgpuEngine.pure.ts: throw on the wrong-engine variant, mirroring the existing wrapXxxTexture override pattern. - wrappedTextureRestore.test.ts: 4 vitests cover rebuild-skip, the new API round-trip, and the throw on non-wrapped target. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
There was a problem hiding this comment.
Pull request overview
This PR attempts to fix context/device-loss handling for externally wrapped textures by preserving InternalTexture identity while allowing hosts to supply replacement native/WebGL/WebGPU handles.
Changes:
- Skips render target wrapper rebuilds when the primary texture has
source === Unknown. - Adds
updateWrappedWebGLTexture,updateWrappedNativeTexture, andupdateWrappedWebGPUTextureAPIs. - Adds unit coverage for wrapped WebGL texture restore behavior.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
packages/dev/core/src/Engines/abstractEngine.pure.ts |
Changes RTW rebuild behavior for Unknown texture sources. |
packages/dev/core/src/Engines/engine.pure.ts |
Adds WebGL wrapped texture handle update API. |
packages/dev/core/src/Engines/nativeEngine.pure.ts |
Adds Native wrong-engine WebGL update override. |
packages/dev/core/src/Engines/thinNativeEngine.pure.ts |
Adds native wrapped texture handle update API. |
packages/dev/core/src/Engines/webgpuEngine.pure.ts |
Adds WebGPU wrapped texture handle update API and WebGL wrong-engine override. |
packages/dev/core/test/unit/Engines/wrappedTextureRestore.test.ts |
Adds unit tests for wrapped texture restore/update behavior. |
Comments suppressed due to low confidence (3)
packages/dev/core/src/Engines/abstractEngine.pure.ts:650
- This source check is too broad for render targets and will skip Babylon-owned multiview render targets: createMultiviewRenderTargetTexture creates an InternalTexture with source Unknown, and WebGLRenderTargetWrapper has a custom _cloneRenderTargetWrapper path to recreate its framebuffer/textures after context loss. It is also too narrow for MRTs because it only checks the first color texture, so a wrapped attachment in another slot would still be rebuilt with format/type -1. Please distinguish externally wrapped RTWs from other Unknown textures and inspect all color attachments.
if (renderTargetWrapper.texture?.source === InternalTextureSource.Unknown) {
packages/dev/core/test/unit/Engines/wrappedTextureRestore.test.ts:65
- This assertion is true before _rebuildRenderTargetWrappers runs because NullEngine.createRenderTargetTexture initializes render-target textures with isReady = true, so the test would still pass if non-wrapped RTWs were accidentally skipped. Please make the rebuild observable (for example by spying on _rebuild or forcing isReady false before the call) so this regression is actually covered.
// The cloned RTW path sets target.isReady = true after _swapAndDie.
expect(regularTexture.isReady).toBe(true);
packages/dev/core/test/unit/Engines/wrappedTextureRestore.test.ts:93
- This only exercises the source !== Unknown branch. Because the implementation uses Unknown as its only wrapWebGLTexture check, the test misses non-wrapped Unknown textures (for example multiview/WebXR-created InternalTextures) that should also throw according to the API contract. Please add a regression case for an Unknown texture not returned by wrapWebGLTexture.
const rt = engine.createRenderTargetTexture({ width: 64, height: 64 }, { generateDepthBuffer: false, generateStencilBuffer: false });
const rtTexture = rt.texture!;
expect(rtTexture.source).not.toBe(InternalTextureSource.Unknown);
expect(() => engine.updateWrappedWebGLTexture(rtTexture, {} as WebGLTexture)).toThrow(/was not produced by wrapWebGLTexture/);
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18469/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18469/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18469/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
Popov72
left a comment
There was a problem hiding this comment.
Same comments as the ones already made by Copilot!
…validation Addresses copilot-pull-request-reviewer's substantive concerns on BabylonJS#18469: - Add InternalTextureSource.External as a positive marker for textures produced by wrapWebGLTexture / wrapNativeTexture / wrapWebGPUTexture, replacing the previous source === Unknown sniff (which is shared with multiview/WebXR/etc. internal textures and would have silently accepted non-wrapped targets in the update API). InternalTexture._rebuild gets an explicit case External: break; mirroring the Unknown fall-through. - Rebuild the render-target wrapper's framebuffer on updateWrappedNativeTexture / updateWrappedWebGLTexture. After context-loss the old framebuffer + depth/stencil renderbuffer are stale; the consumer-supplied new handle is the right moment to recreate them. Skip-everything in _rebuildRenderTargetWrappers was correct -- we genuinely can't rebuild at that point because we don't have a fresh handle yet -- but the rebuild must happen later, inside the update API. - Throw on dimension mismatch (Native; WebGL can't introspect a GPU handle's size so it's documented as the consumer's responsibility) and on multi render-target attachment (both engines). - Reapply samplingMode in update methods so the new resource renders with the consumer-recorded filter rather than the GPU default. - Invalidate engine caches on handle swap: - WebGL: clear cached coordinatesMode/wrap/anisotropic on the InternalTexture and drop any _boundTexturesCache slot pointing at it, so the identity short-circuit in _setTexture (cache[channel] === internalTexture) doesn't skip the rebind. - WebGPU: bump InternalTexture.uniqueId so WebGPUMaterialContext and WebGPUCacheBindGroups detect the resource change and re-resolve bind groups against the new GPUTexture/View. vitests still green (4 wrappedTextureRestore tests + 41 total Engines tests). End-to-end BN coverage (separate PR) updated to render a frame against the rebuilt framebuffer after restore -- still fails pre-fix with the same Matteo stack trace, passes post-fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
…s, pre-validation hoist Self-review surfaced four issues in af80484: 1. _rebuildRenderTargetWrappers skip and updateWrappedXxxTexture walks used rtWrapper.texture, which returns _textures[0]. For a multi-RT with the wrapped texture at index 1+, the skip would not fire (Babylon would rebuild and clobber the wrapped attachment) and the update walk's multi-RT precondition would not trip. Switched both to scan all attachments via textures?.some(...) / textures?.includes(...). 2. The framebuffer-rebuild comment claimed MSAA renderbuffers were rebuilt but the code only rebuilt _framebuffer + _depthStencilBuffer. MSAA with a wrapped color attachment is rarer than multi-RT and not justified as v1 scope; throw on samples > 1 (matches the multi-RT throw pattern). Both WebGL and Native updates now check. 3. _rebuildRenderTargetWrappers "still rebuilds non-wrapped RTWs" test asserted regularTexture.isReady === true, which is already true from createRenderTargetTexture. Now asserts the _hardwareTexture identity is different after the rebuild -- which is what _swapAndDie does to non-wrapped targets but explicitly does not do to wrapped ones (the skip path). 4. Hoisted multi-RT and dimension-mismatch preconditions before any state mutation in updateWrappedWebGLTexture / updateWrappedNativeTexture so a thrown precondition leaves the InternalTexture identity-preserved (proven by the new "does not mutate ... when the multi-RT precondition trips" test). Also tightened test coverage from 4 -> 12 tests, addressing the AI reviewer gap from the original commit (tests asserted what the code obviously set rather than the contract the code is supposed to uphold). New tests cover: the External-vs-Unknown distinction (mimicking a multiview/WebXR InternalTexture with source=Unknown is correctly rejected), multi-RT at index>0 still throws, MSAA throws, sampler-cache and _boundTexturesCache slot clearing. WebGPU additions (separate but related): - wrapWebGPUTexture now reads texture.width/height and records them on the InternalTexture (matching the Native behaviour; previously baseWidth/Height were left at 0). - updateWrappedWebGPUTexture validates new GPUTexture dimensions against the recorded dimensions (GPUTexture exposes .width/.height, so we can). PR description rewrite: honest about the WebGL limitation rather than phrasing it as "consumer's responsibility". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Visualization tests for WebGPU |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
|
WebGL2 visualization test reporter: |
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
WebGL `updateWrappedWebGLTexture` previously threw on `rtw.samples > 1` because the rebuild loop only recreated `_framebuffer` + `_depthStencilBuffer`, leaving `_MSAAFramebuffer` + the multisampled depth buffer stale. The rebuild now branches on samples: - non-MSAA: same inline depth-stencil rebuild as before. - MSAA: zero `rtw._samples` to bypass the helper's idempotency guard, then call `engine.updateRenderTargetTextureSampleCount(rtw, savedSamples)` to rebuild the MSAA framebuffer + color renderbuffer + multisampled depth. Native `updateWrappedNativeTexture`: drop the MSAA throw. The existing rebuild already passes `rtw.samples ?? 1` to `createFrameBuffer`, so the MSAA case is handled correctly when reachable. (Native's `updateRenderTargetTextureSampleCount` is itself a no-op, so MSAA-RTT on Native is unreachable via the public `setSamples` API today; that's a separate engine gap, tracked outside this PR.) vitest: replace the single MSAA-throws test with two tests that assert the rebuild correctly delegates to `updateRenderTargetTextureSampleCount` when `samples > 1` and does NOT delegate when `samples <= 1`. All 13 tests pass. Visualization: add a focused MSAA-RTT smoke test playground snippet (#DY1H5P#0). Renders a rotated red plane to a small 32x32 MSAA RTT, magnifies it to fill the canvas with NEAREST filtering, and pixel-compares against the committed reference. The magnification amplifies any sub-pixel AA signal into a canvas-dominating diff, so MSAA-off regresses loudly. Catches WebGL/WebGPU regressions; not yet enabled on BN side because two existing MSAA-RTT tests in the BN config are already pre-excluded as "Test crashes or hangs on Babylon Native" — MSAA-RTT on Native has known gaps independent of this PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Popov72
left a comment
There was a problem hiding this comment.
Reviewed the updated wrapped texture restore changes. I found a few remaining correctness issues in the new update paths.
The Native engine's updateRenderTargetTextureSampleCount has been a no-op since the engine landed, logging "Updating render target sample count is not currently supported" and returning the current sample count unchanged. RenderTargetWrapper.setSamples(N) -- which is what BJS-side RenderTargetTexture invokes on options.samples AND on the public samples setter -- routes through this method. The silent no-op meant MSAA-RTT on Native was unreachable via the public API: rt.samples = 4 logged a warning and rendered at samples=1 regardless. Implementation needs to recreate both the underlying bgfx texture handle and the framebuffer, because bgfx couples MSAA to the texture creation flags (BGFX_TEXTURE_RT_MSAA_X*) rather than treating MSAA as a separable property of the framebuffer attachment. The flow: 1. initializeTexture re-issues the bgfx texture handle with the new BGFX_TEXTURE_RT_MSAA_X* flag. The Graphics::Texture wrapper identity stays the same -- only the internal handle rotates -- so external InternalTexture references remain valid. 2. _releaseFramebufferObjects + createFrameBuffer with the new samples reissues the framebuffer attached to the new texture handle. Verified with two BabylonNative unit tests (in a separate BN PR): - MsaaRtt.CreateBindClearDispose: low-level engine.createRenderTargetTexture with samples=4 in options. Already worked pre-fix because the option flowed through to bgfx::createFrameBuffer at creation time without involving the setSamples mutation path. Kept as a regression baseline. - MsaaRtt.RenderRotatedPlane: full BJS-level RenderTargetTexture + customRenderTargets + Layer pipeline, exercising the setSamples path that this fix addresses. Pre-fix: the "not currently supported" warning fires during construction and rendering proceeds at samples=1 (silently). Post-fix: warning is gone and rendering succeeds at the requested sample count. Visual-correctness validation (does AA actually anti-alias?) lives in the cross-engine viz suite via playground snippet #DY1H5P#0 -- the same snippet the BJS visualization suite uses for WebGL MSAA-RTT regression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tore # Conflicts: # packages/public/@babylonjs/core/package.json # scripts/treeshaking/side-effects-manifest.json
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
|
WebGL2 visualization test reporter: |
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
…1714 Mark the hasMips=false guards as stopgap workarounds for a bgfx bug rather than D3D11-specific corrections. The same root cause affects D3D11/D3D12/Vulkan and the proper fix belongs in bgfx; tracking issue filed at BabylonNative#1714. Once a fixed bgfx ships in stable BN npm, these two guards should be removed in a follow-up PR.
Per @Popov72 review feedback so selective visualization CI can target the new test correctly.
|
[Responded by Copilot on behalf of @bghgary]
Done in 5293c66 — added |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
1. internalTexture.ts: correct the doc on InternalTextureSource.External. The previous wording (Babylon owns only the wrapping InternalTexture, not the resource) was inaccurate -- WebGLHardwareTexture.release() / NativeHardwareTexture.release() do destroy the wrapped handle on dispose, so ownership transfers to Babylon when the wrapper is created. 2. thinNativeEngine.pure.ts: add an explicit guard at the top of updateRenderTargetTextureSampleCount for External-source textures. Wrapped textures have format/type=-1, so the existing path would throw deeper in getNativeTextureFormat with a confusing error. Throw a targeted error up front instead, and avoid destroying the wrapped handle via initializeTexture.
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
NativeRenderTargetWrapper._framebuffer setter already calls _releaseFramebufferObjects on the old framebuffer before assigning the new one (see Native/nativeRenderTargetWrapper.ts:19-24). The manual calls in updateWrappedNativeTexture and updateRenderTargetTextureSampleCount caused the same bgfx framebuffer handle to be queued for DELETEFRAMEBUFFER twice. Caught in self-review. No behavior change locally on the KHR_materials_volume repro -- bgfx tolerated the double-delete in practice -- but it's a latent issue worth fixing.
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18469/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18469/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18469/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
Visualization tests for WebGPU |
🟢 Memory Leak Test Results13 passed, 0 leaked out of 13 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (13)
|
|
WebGL2 visualization test reporter: |
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
End-to-end test for [BJS #18469](BabylonJS/Babylon.js#18469) — External-source RT wrappers survive a Native device-loss/restore. ## Flow `wrapNativeTexture` on `deviceA` → render red → readback → `DisableRendering` → `UpdateDevice(deviceB)` → next `StartRenderingCurrentFrame` fires BJS `_restoreEngineAfterContextLost` (#18469's skip keeps the wrapped RT intact) → `updateWrappedNativeTexture` → render blue → readback. D3D11-only via `Helpers::CreateDevice()`. `UnhandledExceptionHandler` calls `std::quick_exit(1)` (matching `Tests.JavaScript.cpp` et al.) so a BJS regression that rebuilds the External wrapper terminates loudly. Requires `@babylonjs/core` ≥ 9.9.1. ## Local verification - Positive: PASSES; BJS logs `"WebGL context successfully restored."` - Negative: stubbing #18469's External-source skip out of local `babylon.max.js` → test exits 1 on uncaught `Unsupported texture format` inside `_rebuildRenderTargetWrappers`. [Created by Copilot on behalf of @bghgary] --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bugs
Wrapped texture restore after context loss
Wrapped textures (
wrap{WebGL,Native,WebGPU}Texture) carry an opaque external handle with unknown format. On context loss,_rebuildRenderTargetWrappers → createRenderTargetTexture(format=-1)throws on Native and silently clobbers the consumer's_hardwareTextureon WebGL.Native
updateRenderTargetTextureSampleCountno-opThe Native impl has been a no-op since landing, logging
"Updating render target sample count is not currently supported".RenderTargetWrapper.setSamples()— the pathRenderTargetTextureuses for bothoptions.samplesand the.samplessetter — routes through it, so MSAA-RTT on Native was unreachable via the public API. The low-levelengine.createRenderTargetTexture(size, {samples})path worked because it plumbs straight intobgfx::createFrameBufferat creation time.Fix
InternalTextureSource.Externalmarker, set bywrap*Texture. Guards key onExternalso they apply only to wrapped textures._rebuildRenderTargetWrappersskips wrapped RTWs. The framebuffer rebuild is deferred to step 3 where we have the new handle.updateWrapped{WebGL,Native,WebGPU}Texturerepoints a wrapper at a fresh handle, preservingInternalTextureidentity. Pre-validates (External marker, dimensions where introspectable, multi-RT / depth-stencil-texture throws), reappliessamplingMode, rebuilds framebuffer + depth/stencil. Forrtw.samples > 1the MSAA branch delegates toupdateRenderTargetTextureSampleCount. Invalidates engine caches (WebGL sampler params +_boundTexturesCache; WebGPU bumpsuniqueIdso material context / bind groups re-resolve).updateRenderTargetTextureSampleCountreal impl inthinNativeEngine.pure.ts. bgfx couples MSAA to texture creation flags, so mutating samples requires re-issuing both the texture (initializeTexturerotates the bgfx handle while preserving theGraphics::Texturewrapper identity) and the framebuffer (_releaseFramebufferObjects+createFrameBuffer). Idempotent onrtw.samples === samples.MipLevels > 1on textures withSampleDesc.Count > 1; D3D12/Vulkan have equivalent rules. bgfx's D3D11 backend reuses one descriptor for both the MSAA target and the resolve target and doesn't resetMipLevelsbetween them, so requesting MSAA on a mipped RT crashesCreateTexture2DwithE_INVALIDARG. The trigger in practice is the glTF transmission helper (transmissionHelper.ts:398) which creates anopaqueSceneTextureRTT withgenerateMipmaps: trueand immediately setssamples = 4. Workaround in BJS: forcehasMips = falsewhensamples > 1in both the construction (_createInternalTexture) and mutation (updateRenderTargetTextureSampleCount) paths, taggedTODO(bgfx-msaa-mips). The proper bgfx-side fix is tracked in BabylonNative#1714; the guard should be removed once a fixed bgfx ships in stable BN npm.Tests
16 vitests in
wrappedTextureRestore.test.tscovering the guard, the API round-trip, multi-RT throw + half-mutation invariant, depth/stencil-texture throw, sampler-cache + binding-cache invalidation, and the MSAA-rebuild delegation (with/withoutsamples > 1).Visualization: playground
#DY1H5P#0— rotated red plane → 32×32 MSAA RTT, magnified with NEAREST so samples=1 regressions are canvas-dominating. Reference atReferenceImages/msaa-rtt-samples4-magnified.png.End-to-end Native coverage queued as a follow-up BabylonNative PR (bumps
@babylonjs/corein the same PR once this ships).Local repro confirmation
Reproduced the CI
BGFX FATAL 0x80070057 at renderer_d3d11.cpp(4601)locally on Win32 D3D11 Debug, running the exact BN config testGLTF Extension KHR_materials_volume with attenuationagainst an unfixed BJS UMD. With the guard in place the same test runs to completion (Run complete. ran=1 passed=1). Also verifiedcreateDefaultEnvironmentalone does not touch the MSAA path — the bug is exclusive to the transmission helper.[Created by Copilot on behalf of @bghgary]