feat(#573)!: remove chroma-key everywhere — ABI v4 (runtime half) - #577
Merged
Conversation
Generalize the D3D11 client's ADR-029 transparent present (client owns the
DComp present on its own HWND because a process can't present cross-process
onto another's window) into a reusable, render-API-agnostic helper so the
D3D12/GL IPC clients can close their see-through gap without chroma-key.
- New comp_d3d_transparent_present.{h,cpp}: imports the service's shared
NT-handle texture + service->client fence and presents on the app HWND
(wait -> copy -> Present -> Commit). Reuses a passed ID3D11Device (D3D11
client) or creates its own (D3D12/GL/VK) — the present is pure D3D11+DComp
on shared handles, independent of the app's render API. C-callable shim so
the .c GL client can use it.
- CMake: build it in the shared D3D block; link d3d11/dxgi/dcomp there.
- comp_d3d11_client.cpp: consume the helper (−131 lines), behavior-identical
(passes its own app_device). This is the Windows verification anchor.
First step of removing chroma-keying everywhere (#573). VK over IPC already
done via #568; only D3D12/GL clients + GL no-interop in-process remain to
close before the ABI v4 deletion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The P0.1 helper was authored on macOS and never compiled. MSVC rejected it
on three counts (all in the new comp_d3d_transparent_present.{h,cpp}):
- header used uint64_t/uint32_t without <stdint.h> (xrt_handles.h doesn't
pull it in) -> C2061 syntax error, which in turn made the .cpp definition
look like an extern "C" overload (C2733).
- the own-device branch called xrt::compositor::client::createDevice(), but
createDevice lives in xrt::auxiliary::d3d::d3d11 (d3d_d3d11_helpers.hpp)
-> C3083/C2039/C3861.
Full runtime build is now clean (=== ALL DONE ===). D3D11-client behavior is
unchanged (it still passes its own app_device; the own-device branch is only
reached by the not-yet-wired D3D12/GL clients).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(P0.2)
Close the D3D12 / GL forced-IPC see-through gap without chroma-key by reusing
the P0.1 render-API-agnostic presenter, mirroring the D3D11 client exactly.
Both pass NULL device so the helper stands up its own D3D11 device (the present
is pure D3D11 + DComp on the shared handles, independent of the app's render
API).
- comp_d3d12_client.cpp: + `transparent` field + destructor cleanup; after fence
init in _create, call get_transparent_output(+_fence) RPCs and create the
presenter (NULL device); present after both successful layer_commit paths
(semaphore + plain).
- comp_gl_win32_client.{c,h}: same, but the parent GL client owns layer_commit,
so save its fn ptr and wrap it (present after a successful base commit) — only
when a presenter was created. Forward-declared the presenter in the C header.
- tests/CMakeLists.txt: the d3d12 + (Windows) opengl client unit tests now
reference the IPC bridge symbols, so link `ipc_client` (as the d3d11 test
already does).
VK client intentionally skipped — Android #568 already covers VK transparency
(service-presents-to-shared-Surface). Full runtime build clean (=== ALL DONE ===).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the last in-process see-through gap so chroma-key can be deleted: when WGL_NV_DX_interop2 is unavailable (near-extinct hardware), the GL native compositor previously fell back to an OPAQUE SwapBuffers. Add a glReadPixels → D3D11 dynamic-texture upload → DComp blit readback path instead, so α=0 holes still reveal the live desktop. - Refactored the proven interop setup into gl_setup_dcomp_common (D3D11 device + PREMULTIPLIED composition swapchain + DComp bind + blit pipeline), shared by both the interop and readback setups; and gl_dcomp_blit_srv_present, shared by both present-frame paths. Interop path verified unchanged on RTX 3080 (still logs "DComp present path active … interop transit"). - Readback path weaves into a DEDICATED RGBA8 GL texture+FBO — NOT FBO 0. The window's default framebuffer has no usable alpha, so glReadPixels returned A=1 and the see-through holes came out opaque black; a real RGBA target preserves the premultiplied alpha. glReadPixels row 0 = GL bottom = the interop transit's orientation, so the existing blit shader's V-flip is correct unchanged. - DISPLAYXR_GL_FORCE_READBACK=1 forces the readback path on interop-capable GPUs for verification. Per-frame readback cost logged once. Verified on the Leia box (in-process, DISPLAYXR_TRANSPARENT_BG=1): interop regression + forced readback both present cleanly; readback see-through Leia-eyeball-validated (live desktop through the holes, de-occlusions correct). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lay half) Phase 1 of removing the chroma-key transparency mechanism. ABI-breaking: XRT_PLUGIN_API_VERSION_CURRENT 3→4; XR_EXT_win32_window_binding SPEC_VERSION 7→8. True transparency (alpha-capable swapchain + transparent present, landed in P0) is now the sole path. Display-processor interfaces (the ABI contract): - Removed set_chroma_key from all five DP vtables (base/vk, d3d11, d3d12, gl, metal) — a mid-vtable slot, so every following slot shifts; renumbered every offset static_assert + the size assert in each header. - Added set_transparent_background to the D3D12 and GL DP vtables (d3d11 + vk already had it; metal needs none — its compositor sets drawable clear-alpha itself). This carries the transparency *enable* that on d3d12/gl had been overloaded onto set_chroma_key. - Deleted the per-API set_chroma_key wrapper helpers; added d3d12/gl set_transparent_background wrappers. Runtime: - Dropped chroma_key_color from xrt_session_info (xrt_compositor.h), all five compositor create signatures + the oxr session-gfx native plumbing + the XR_EXT read in oxr_session.c. - The four in-process compositors (d3d11/d3d12/gl/vk) now call set_transparent_background(enabled, client_presents=false) instead of set_chroma_key — client_presents=false preserves the prior compose-under-bg behavior exactly (the old set_chroma_key carried no client_presents, so the DP defaulted to compose-under-bg). The GL DComp transparent present is independent and keeps working. - Deleted the D3D11 service svc_chroma_key_* pass (shaders, ChromaKeySvcConstants, init/ensure/execute) + the ck_* resource fields. - Removed chromaKeyColor from the extension struct. - Bumped the oxr_plugin_stub ABI-drift static_assert to v4. sim_display (in-repo v4 test double): deleted all five set_chroma_key stubs. Verified on the Leia box: full runtime build clean (all ABI static_asserts pass); `displayxr-cli selftest` PASSES with the freshly-built v4 sim_display (the v3 Leia + v3 sim plug-ins are correctly rejected, negotiate -17). The leia-plugin chroma deletion + d3d12/gl set_transparent_background impls are the remaining lockstep half (separate repo) before any tag/release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dfattal
force-pushed
the
chore/remove-chroma-key
branch
from
June 14, 2026 10:00
151e0f3 to
088b53f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runtime half of #573 — the clean-slate removal of chroma-keying. True transparency (alpha swapchain + transparent DComp present + DP compose-under-bg / alpha-gate) fully replaces the chroma-key fallback. Coordinated ABI v3→v4 break, lockstep with
displayxr-leia-pluginPR (feat/remove-chroma-key-573).What's in here
set_chroma_keyremoved from all 5 DP vtables;set_transparent_backgroundadded to D3D12 + GL DP vtables (d3d11/vk already had it);chroma_key_color/chromaKeyColordropped fromxrt_session_info+ 5 compositor signatures + the oxr gfx chain;XR_EXT_win32_window_bindingSPEC_VERSION 7→8; D3D11-service chroma pass + sim_display stubs deleted.XRT_PLUGIN_API_VERSION_CURRENT3→4.Validation
displayxr-cli selftestpasses only with a v4 plugin (v3 plugins →negotiate -17, ABI reject working).Lockstep
Do not release the v4 runtime without the v4 Leia plugin — a v4 runtime rejects a v3 plugin. Paired with leia
feat/remove-chroma-key-573; release together (runtime v1.18.0 + leia v1.8.0).Closes #573.
🤖 Generated with Claude Code