Transparent background fill for StageStyle.EXTENDED windows

[crschnick]

This is a small experiment for the request for a transparent background fill for extended windows. In reference to a discussion on the mailing list a few months ago: https://mail.openjdk.org/pipermail/openjfx-dev/2025-October/056849.html

This treats EXTENDED windows just as UNIFIED windows on Windows. There is still the lingering issue with UNIFIED something resulting in a blank window (https://bugs.openjdk.org/browse/JDK-8154847), but there is a workaround with Dprism.forceUploadingPainter=true

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/2045/head:pull/2045
$ git checkout pull/2045

Update a local copy of the PR:
$ git checkout pull/2045
$ git pull https://git.openjdk.org/jfx.git pull/2045/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 2045

View PR using the GUI difftool:
$ git pr show -t 2045

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/2045.diff

[bridgekeeper]

👋 Welcome back crschnick! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[mstr2]

How is this supposed to work? If I set the scene background to Color.TRANSPARENT, all I get is a window with a black background (both with the presenting painter and the uploading painter). Same as with StageStyle.UNIFIED.

[crschnick]

Last time I tested the EXTENDED style, you always got a white background layer even if the scene background and other stuff was transparent.

Maybe transparent background is the wrong naming, the point of this PR is to get the raw window frame background and not a white one. The window frame can be light, dark, or anything else if you customize it via native calls.

E.g. if you call DwmSetWindowAttribute with DWMWA_SYSTEMBACKDROP_TYPE via a native call, you can also use something like the Mica theme for the window. Then, with this PR, you get the actual Mica background for the whole window. That is/was the appeal of the UNIFIED stage style, which would also be nice to have for the EXTENDED style

[mstr2]

I think this enhancement would be more compelling if it actually gave you the translucent/acrylic background material.

[beldenfox]

I am just about to submit a PR for JDK-8375578 which enables translucent system backdrops on macOS and Windows 11. I will get that PR submitted in a day or two.

[crschnick]

@mstr2 In general, do you see any potential problems with this change? Or would this be ok to be added like this

[mstr2]

@mstr2 In general, do you see any potential problems with this change? Or would this be ok to be added like this

Changing an internal implementation detail for one particular OS so that you can later do something that we explicitly don't support is an answer for the wrong question. There's no API in JavaFX to interact directly with the native window implementation, and that's for a good reason. If people start depending on a particular implementation detail, the system will become progressively harder to maintain.

Let's do it the other way around, and start by asking what the goal should be. I understand that you want to have translucent window backdrops; people have been asking for this feature for ages, and we've discussed it several times. But we need an API and a specification for that, and then support it indefinitely on at least Windows and macOS.

If I really needed to make something work for an application and I needed it now, I'd probably build a custom version of JavaFX with the understanding that I'm doing this at my own risk.

[crschnick]

Ok maybe I focused on the wrong thing in my reasoning. Even without customizations, now that the window will follow the native dark mode setting with scene preferences, the window frame might be dark. Right now, if your scene background has some transparency, the EXTENDED stage style will still fill the background with white, meaning that you can't use the dark frame as a background.

For example, this is how an app that has some transparency in the fill and node background colors looks like with extended mode when the system dark mode is enabled:

The right sidebar node has a completely transparent background, yet the window background is still filled with white. Even if the window frame is dark.

This is how it looks like with this PR:

The native window background is now visible.

[mstr2]

Looking at the code in GlassScene, the clear color is always set to white if it isn't opaque. Wouldn't it make more sense to more gracefully derive a non-transparent version of the specified color instead?

[mstr2]

I think we need to have an additional piece of information in GlassScene to determine the clear color: the scene's color scheme. Otherwise we can't recover an "opaque" version of the scene's fill color.

Let F be the scene fill color: F = (Fr, Fg, Fb, Fa)
Let B be the scene color scheme (either black or white): B = (Br, Bg, Bb)
Then the opaque clear color is: (Cr, Cg, Cb, 1) = (Fr+Br(1−Fa), Fg+Bg(1−Fa), Fb+Bb(1−Fa), 1)

Let's suppose I specify Color.TRANSPARENT as the scene fill, which is (0, 0, 0, 0). With a black color scheme, the formula yields the clear color (0, 0, 0, 1), and with a white color scheme, it yields the clear color (1, 1, 1, 1).