hal/vulkan: Accurately map all formats except fp16 to use the non-linear sRGB color space

[MarijnS95]

Connections
Fixes #8076

Description

In a quest to deduplicate reported texture formats in #3227 this was hacked in to be based on color spaces (having the same format listed for multiple color spaces is how duplicates exist to begin with), following a link to some swapchain code that forcibly uses linear scRGB (extended sRGB) for a surface configuration with fp16 texture format. By accident it seems also 16-bit UNORM/SNORM formats and even the A2R10G10B10_UNORM_PACK32 format were placed inside the linear scRGB match arm. Besides mismatching with the forced color space used at swapchain creation, this disallows using higher bit-depth in regular non-linear sRGB output. In particular 10-bit is popular for modern monitors, which remain in (non-linear) sRGB while HDR output is disabled.

According to the linked Vulkan reports database the existing format pairs with EXTENDED_SRGB_LINEAR_EXT have barely any coverage, to the point where R16G16B16A16_SNORM doesn't seem to be supported at all with anything other than SRGB_NONLINEAR_KHR.

Testing
No testing yet.

Squash or Rebase?

Rebase

Checklist

• Run cargo fmt.
• Run taplo format.
• Run cargo clippy --tests. If applicable, add:
  • --target wasm32-unknown-unknown
• Run cargo xtask test to run tests.
• If this contains user-facing changes, add a CHANGELOG.md entry.

[MarijnS95]

In an ideal world users of wgpu are allowed to choose a format and color space combination, but I assume simply no API exists for that yet which is why this is hardcoded to force linear scRGB on fp16?

wgpu/wgpu-hal/src/vulkan/device.rs

Lines 505 to 511 in 04a3401

	let color_space = if config.format == wgt::TextureFormat::Rgba16Float {
	// Enable wide color gamut mode
	// Vulkan swapchain for Android only supports DISPLAY_P3_NONLINEAR_EXT and EXTENDED_SRGB_LINEAR_EXT
	vk::ColorSpaceKHR::EXTENDED_SRGB_LINEAR_EXT
	} else {
	vk::ColorSpaceKHR::SRGB_NONLINEAR
	};

[Wumpf]

Yeah I think the surface APIs should expose color spaces on surfaces similar to how Vulkan does, i.e. first allow the adapter to advertise which format+space pairs are available and then allow surfaces to be created in a way that spells out what space (& encoding) + format the compositor will expect values to be that end up in the surface.

See also:

• Well-defined HDR surface support #2920

What makes this tricky is that WebGPU exposes the color space of a surface/canvas: https://www.w3.org/TR/webgpu/#canvas-configuration
As per definition all color spaces are extended range. But while surface configuration allow specifying the color space, it doesn't seem to specify the encoding/transfer function. That part confuses me a bit since that means that an important puzzle piece seems to be missing: in the example here we'd want color space to be srgb in either case, but sometimes in linear and sometimes with the OETF applied (i.e. SRGB_NONLINEAR).
Digging a bit deeper, CSS defines linear srgb as a separate color space: https://drafts.csswg.org/css-color/#predefined-sRGB-linear. And as I understand this is the same property that WebGPU is referring to in its canvas configuration.
Therefore I think we'd have to assume that RGBA16F on the web with the surface set to srgb is actually vk::ColorSpaceKHR::EXTENDED_SRGB_NONLINEAR_EXT , meaning that the linked code makes it inconsistent with WebGPU 😱

[Wumpf]

in other words if we want to start by mirroring WebGPU (and we absolutely should, to make things portable at least), then

wgpu/wgpu-hal/src/vulkan/device.rs

Lines 505 to 511 in 04a3401

	let color_space = if config.format == wgt::TextureFormat::Rgba16Float {
	// Enable wide color gamut mode
	// Vulkan swapchain for Android only supports DISPLAY_P3_NONLINEAR_EXT and EXTENDED_SRGB_LINEAR_EXT
	vk::ColorSpaceKHR::EXTENDED_SRGB_LINEAR_EXT
	} else {
	vk::ColorSpaceKHR::SRGB_NONLINEAR
	};

is actually wrong and it should use EXTENDED_SRGB_NONLINEAR_EXT if available.

[Wumpf]

Addendum / looking at the sources step by step:

WebGPU points to https://html.spec.whatwg.org/multipage/canvas.html#predefinedcolorspace
Which in turn points to https://drafts.csswg.org/css-color/#valdef-color-srgb
Which explicitly states the transfer function per color space (depending who you ask color space is independent of transfer function since that's just an encoding detail, but they bundle that here!)

Therefore,

wgpu/wgpu-types/src/lib.rs

Line 7461 in 3efd640

Srgb,

should strictly imply non-linear encoding (we can add a linear variant for native only!), should really doc & link that up better.
Therefore, we should use vk::ColorSpaceKHR::EXTENDED_SRGB_NONLINEAR_EXT as previously stated :)

[Wumpf]

Might be related to

• Vulkan color space is different when rendering to swap chain (sRGB) #4842

@valaphee also made the observation that the hardcoded "I guess float means linear extended srgb, everything else non-linear srgb": #4842 (comment)

[Wumpf]

despite lengthy comments above I think this change is a strict improvement and is correct to point to the (imho even more wrong) hardcoded color space bit.

Changelog entry would be nice either way and I'd like to hear your thoughts on what I diged up :)

[jimblandy]

This presentation, made to the W3C WebGPU committee in August, was great background on HDR:

https://docs.google.com/presentation/d/13Gu_ZhNdMzKu285VLP3CNSjRlheo0bZsCHGml6FzlKY/edit?slide=id.p#slide=id.p

[cwfitzgerald]

Yeah lets just get a changelog entry and we can land. This is better than before.