Well-defined HDR surface support

[kpreid]

Is your feature request related to a problem? Please describe.
Currently, on macOS/Metal, if you use a SurfaceConfiguration with format TextureFormat::Rgba16Float, you can render color component values greater than 1.0 — the HDR functionality of the compositor and Apple laptop displays is activated. (I don't have other platforms readily available to test on.)

However, this is not sufficient for good HDR rendering; some information is needed by the application.

Describe the solution you'd like

I'm not greatly familiar with matters of HDR displays myself, but after some discussion with cwfitzgerald and Ralith on the Rust Game Dev Discord, I believe the following API elements are needed:

• Communication from the compositor/display to the application of what the largest displayable values are (since this can no longer be presumed to be 1.0 or 255), so that the application can decide how to work within that range rather than experience unknown clipping.
• Defining what color space primary colors are being used (e.g. “always the same as the sRGB primaries” would be fine, but it needs to be specified), either as a fixed documented “this is what the API expects”, application's choice, or (least preferable) reporting what the surface wants. (This would also be useful functionality to non-HDR renderers, to nail down what should be expected rather than “eh, it's platform-dependent”.)

Describe alternatives you've considered
The status quo is useful but limited.

Additional context

Quick demonstration that HDR does already work when I tweaked the surface configuration in my application, producing brighter-than-nominal-white results (necessarily a photograph of a monitor!):

However, the sphere is still flat white, whereas it “should” have some detail — illustrating that high-dynamic-range isn't infinite-dynamic-range. Knowing the maximum renderable luminance is needed to be able to improve on this.

[jinleili]

The available range of pixel values will change as the current screen brightness changes and as the user drags the window between screens. wgpu currently has no mechanism to observe to and actively notify the app of changes in the available range, it may eventually have to rely on wsi libs such as winit to achieve this.

[superdump]

This sounds like it needs an HDR calibration step as seen in many games where you try to adjust some slider until you can only just distinguish between two values. Does anyone have information on how those calibration screens are implemented and how to apply necessary rescaling or so to fit within that range? Is it a case of tonemapping to that range instead of just to 0.0-1.0?

[kpreid]

@superdump The point of those calibration screens (as I understand it; I haven't written or studied one) is to adjust the tone mapping to the display properties and lighting conditions so that things in game scenes are visible when expected. These are things that the window system cannot detect (on current-day hardware, at least), because they depend on the light in the room, the user's vision, and deficiencies in the display or its settings. Furthermore, these issues exist even in rendering to SDR output.

On the other hand, the maximum value before clipping is

1. unique to HDR output (because in the SDR world it is by convention 1.0 or 255), and
2. something the window system necessarily already knows.

Yes, you could extract an approximation of it by asking the user ("when does this white shape disappear into the different-shade-of-white background?") but why bother them with squinting at that, and hope they give you an accurate answer? Better to have fewer knobs to get wrong (producing an either clipped or unnecessarily dim image), and spend the user's time on something that is more subjective and beneficial to them ("what is dark-but-not-too-dark?"). Likely you can get the best results by incorporating subjective calibration and the maximum value into your tonemapping function.

And whether or not you decide to go down that path of user-supplied calibration — that's not going to be part of wgpu's API, regardless, since it's about adjusting tonemapping parameters after displaying a prompt and sample scene to the user, all of which are things the application defines, not wgpu.

[superdump]

Right, and it seems some displays and OS APIs can provide information about what the display claims to support. A problem could be there are standards like DisplayHDR 400 that are practically useless in terms of high dynamic range, or higher DisplayHDR profiles that do not have full array local dimming with many zones, just edge lit and still meet the requirements but offer a far inferior HDR experience. Not that a calibration screen would necessarily catch that example.

Anyway, I get your point that discussion here should focus on what wgpu might need to implement and expose. I was trying to understand how to implement HDR through to the display in order to understand what needs to be implemented where.

I see that on macOS it seems that when creating a Metal surface, as in the link noted by jinleili, you can inform the API that you will provide extended dynamic range content, and for rendering purposes where calculations are conducted in a linear colour space, one can set a colour space by choosing the appropriate primaries for the display and using a linear transfer function https://developer.apple.com/documentation/coregraphics/cgcolorspace#1692873 , and the pixel format. So aside from the pixel format, it seems on macOS one would want to be able to configure the colour primaries and transfer function for the surface.

Windows seems to, from a skim of https://learn.microsoft.com/en-us/windows/win32/direct3darticles/high-dynamic-range , target scRGB, which is apparently sRGB colour primaries with a linear transfer function. I don't see any way to configure say DCI-P3 primaries.

No idea about Linux with X.org or wayland. Nor other platforms (web, android, iOS, etc).