Some observations regarding SDR content playback in mpv (Fedora KDE)

[FTKV]

Please allow me to share some observations regarding SDR content playback in mpv, its configuration settings, and the interaction with Wayland.

My system setup is a laptop running an up-to-date Fedora 44 with KDE and Wayland, an Intel CPU with integrated GPU, and up-to-date Mesa, Intel Media Drivers, FFmpeg, and mpv 0.41. The laptop display features a near-P3 D65 color gamut and a native transfer function close to a pure 2.2 gamma. Therefore, I consider the options target-prim=display-p3 and target-trc=gamma2.2 to be a reasonably acceptable approximation of the display's capabilities (along with target-contrast=1200 close to specs and tests and treat-srgb-as-power22=no to properly distinguish the sRGB EOTF from pure gamma 2.2). This is a rather typical, common configuration without any exotic specifics. The core options of my mpv config are as follows:

vo=gpu-next
gpu-api=vulkan
hwdec=vulkan,vaapi
gpu-context=waylandvk

Since most SDR video content is encoded with BT.709 primaries and a BT.1886 EOTF, it needs to be adapted for correct playback on my laptop (accurate to the specified approximation)—both the color space from BT.709 to Display P3 D65, and the EOTF from BT.1886 to pure gamma 2.2. Thus, the "ideal pipeline" of transformations should look like this:

data encoded with BT.709 primaries and inverse BT.1886 EOTF -> forward BT.1886 EOTF -> from BT.709 to Display P3 D65 transformation -> inverse pure gamma 2.2 -> display with forward pure gamma 2.2

It seems that this is exactly what I can achieve by configuring mpv according to Option A.

Option A:

target-colorspace-hint=no
target-contrast=1200
target-prim=display-p3
target-trc=gamma2.2
treat-srgb-as-power22=no

mpv stats info:

In this specific case, I can be certain that mpv correctly performs both transformations—both the color space conversion (preventing oversaturation in reds and greens, which happens when BT.709 content is mapped directly onto a P3 display without conversion) and the transfer function mapping. This is indeed confirmed by the visual result. So, the pipeline looks roughly like this:

data encoded with BT.709 primaries and inverse BT.1886 EOTF -> mpv applies forward BT.1886 EOTF -> mpv applies from BT.709 to Display P3 D65 transformation -> mpv applies inverse pure gamma 2.2 -> Wayland has nothing to do -> display with forward pure gamma 2.2

For sRGB content, it would map accordingly:

data encoded with BT.709 primaries and inverse piecewise sRGB EOTF -> mpv applies forward piecewise sRGB EOTF -> mpv applies from BT.709 to Display P3 D65 transformation -> mpv applies inverse pure gamma 2.2 -> Wayland has nothing to do -> display with forward pure gamma 2.2

Option A seems ideal, but it has its downsides—specifically, I cannot assign a display ICC profile in the operating system's Display Configuration in this case. Option A works perfectly when mpv handles all transformations on its own, while Wayland assumes the display is a default one with BT.709 primaries and an sRGB EOTF, thereby passing the processed data from mpv directly to the display without further alterations. However, if I add the display's ICC profile (P3 D65 and pure gamma 2.2) under this setup, it appears that Wayland performs an additional BT.709 to Display P3 D65 transformation, causing the colors to become undersaturated.

Nonetheless, correct reproduction can be easily achieved with an ICC profile active via Option B.

Option B:

target-colorspace-hint=yes
target-contrast=1200
target-prim=display-p3
target-trc=gamma2.2
treat-srgb-as-power22=no

mpv stats info:

The playback appears correct again at first glance, but some questions remain. First, why do the display properties now report Transfer: linear and Format: rgba16? Second, under Option B, the target-contrast=1200 option now affects not only BT.1886 content but also sRGB and any other content, and it does so quite aggressively (even though the black point in the ICC profile is a perfect zero). Under Option A, the target-contrast=1200 option only affects BT.1886 content and acts gently, exactly as intended. Third, for some reason, there is no difference between the target and source values of the target-colorspace-hint-mode option.

An interesting but unacceptable result is obtained if the target-colorspace-hint-strict=no option is added—Option C.

Option C:

target-colorspace-hint=yes
target-colorspace-hint-strict=no
target-contrast=1200
target-prim=display-p3
target-trc=gamma2.2
treat-srgb-as-power22=no

mpv stats info:

In this case, Transfer shows gamma2.2 (just like in Option A), but Format becomes rgba16 (just like in Option B). Also, there is still no difference between the target and source values of target-colorspace-hint-mode. The issue with Option C is that an inverted gamma is applied twice—presumably by both mpv and Wayland—causing the image to become catastrophically overexposed and clipped.

It is also worth noting that in my software stack, the icc-profile-auto and icc-force-contrast options have absolutely no effect in any of these cases.

Could you please advise which option is better to stick with for SDR content—Option A or Option B? Or perhaps the configuration needs to be tweaked into a different approach altogether? On one hand, with Option A, all conversions are executed solely by mpv, ensuring their high quality. On the other hand, the mpv manual states that target-colorspace-hint=<auto|yes> is highly recommended to use, which leans in favor of Option B...

Have you noticed any unintended behavior or bugs here?

Thank you.

[kasper93]

mpv 0.41

Have you noticed any unintended behavior or bugs here?

Before I read this, could you retest with mpv and libplacebo built from master. There were significant changes to all this in not yet released builds.

[kasper93]

Recently, support for Profile 7 MEL was added to mpv.

Profile 7 MEL was supported for at least 4 years at this point.

Is FEL support next in line? Will it only implement RPU processing for FEL, or will it handle the residual signal as well, allowing for the reconstruction of the 12-bit master?

Yes, FEL is implemented and will be merged soon, see #17932

Given all of this, the task of figuring out exactly how DV formats the signal for TV-led mode doesn't seem insurmountable. On the contrary, Player-led seems more overhead-heavy for the player and harder to implement, whereas for TV-led, you only need to multiplex the signal with the RPU and send it over HDMI—the TV will handle all the remaining processing.

Cool. Now tell me how to do it and I will be happy to do it. (hint: you can't just "multiplex" HDMI signal in user application)

[FTKV]

Profile 7 MEL was supported for at least 4 years at this point.

I didn't mean the fallback: #14692

Yes, FEL is implemented and will be merged soon, see #17932

Only RPU? Or residual signal for 12-bit reconstruction as well?

Cool. Now tell me how to do it and I will be happy to do it. (hint: you can't just "multiplex" HDMI signal in user application)

I'm not giving you advice, just thinking out loud. Could you explain in more detail what the specific issues with TV-led are here on PC?

[Headcrabed]

I'm not giving you advice, just thinking out loud. Could you explain in more detail what the specific issues with TV-led are here on PC?

It requires an special closed os-specific API to switch your TV to that mode, and we don't know what is the API and how to use it.

[kasper93]

I didn't mean the fallback: #14692

This change is more insignificant that you think.

Only RPU? Or residual signal for 12-bit reconstruction as well?

Everything.

[FTKV]

This change is more insignificant that you think.

If you say so. But do you really mean to say that full MEL support has been working for over four years now, rather than just a fallback to HDR10?

Everything.

Great.

Which specific metadata from the RPU does mpv actually account for tone-mapping?

[Headcrabed]

For wayland+KDE, to allow full color management pipeline:

1. Set color profile to built-in in KDE display settings(if you don't have an ICC file) or just set that to your ICC if you have one dedicated to your display.
2. mpv config:

vo=gpu-next
gpu-api=vulkan
treat-srgb-as-power22=both
sdr-adjust-gamma=yes

treat-srgb-as-power22 should set to both, since KDE's Kwin is already treating all sRGB transfer function to pure power 2.2.
sdr-adjust-gamma=yes to enable color space conversion for SDR content.
target-colorspace-hint and target-colorspace-hint-mode could be just kept as default value. It should work well in such environment.

[FTKV]

@kasper93

[kasper93]

I see that for target-prim, only standard colorspaces are allowed, whereas for target-gamut, custom coordinate values can also be specified. Is this restriction due to the fact that compositors only accept data in standard container colorspaces on the "input" side? Or is this temporary, and will the ability to set custom values be added for target-prim as well?

Target primaries is named set of primaires, you can send bt.2020 over HDMI, inside that you can gamut map anything as you want. I don't know what do you expect to change.

[FTKV]

Target primaries is named set of primaires, you can send bt.2020 over HDMI, inside that you can gamut map anything as you want. I don't know what do you expect to change.

This is exactly what I want to clarify.

target-gamut accepts arbitrary xy coordinates and predefined named standard colorspaces, while target-prim only accepts predefined named standard colorspaces.

Is target-prim intentionally restricted to standardized colorspaces (BT.709, BT.2020, P3, etc.), while target-gamut is intended as the arbitrary gamut? Or is this difference is temporary, and will the ability to set arbitrary xy coordinates be implemented for target-prim as well?

[kasper93]

intentionally restricted

I wouldn't call it intentionally restricted. It's the configuration that is on the wire. While the target-gamut is configuration that our renderer limits and map the colors to. You can think about this as target-prim is "HDMI colorspace" while target-gamut is "display colorspace".

For example you might want to configure you PJ to receive bt.2020 signal, but if it doesn't have full 100% coverage, you can limit the colors to real gamut of the display.

[FTKV]

You can think about this as target-prim is "HDMI colorspace" while target-gamut is "display colorspace".

Yes, I understand this technique: the RGB triplets are encoded in the container colorspace ("HDMI colorspace"), but the set of their values is restricted by another colorspace, such as the "custom EDID colorspace" ("display colorspace"). This is analogous to how typical HDR content today is delivered in a BT.2020 container, but the actual colors are restricted to the P3 colorspace.

However, I have a couple of thoughts regarding its implementation.

First, I doubt it makes sense for this to be triggered in my case described above—where the Built-in (EDID) setting is enabled instead of an ICC profile. After all, this is a SDR display with gamma 2.2 (which mpv reads correctly). If it were an HDR display, it would make sense to encode the content into a BT.2020 container colorspace but restrict the colors by the custom EDID colorspace. But in the case of an SDR display, I would expect the data to be encoded by mpv directly into the custom EDID colorspace and sent to KWin in that form, so that KWin wouldn't have to perform any additional conversions. Of course, this would only work if KWin is capable of accepting input data encoded in a custom colorspace rather than a standardized one.

Second, I disagree with "raw primaries are clamped inside the container primaries." Look, we can think of the container colorspace and the custom EDID colorspace as triangles on the $xy$ plane, with their vertices defined by the respective primaries. The set of allowed colors can be thought of as the intersection of these two triangles. The situation is trivial when the custom colorspace triangle is entirely inside the container colorspace triangle—then we can simply clamp all RGB triplets by the custom colorspace. However, if the custom colorspace triangle does not fit entirely within the container colorspace triangle—which is exactly the case with my display's EDID primaries—then the intersection of the two triangles becomes a quadrilateral, a pentagon, or a hexagon. In this scenario, we would need to clamp all RGB triplets by both the custom colorspace and the container colorspace. But you CANNOT clamp the primaries themselves! Doing so makes no sense: clamped primaries still define only a triangle, whereas here the allowed color gamut is restricted by a 4-, 5-, or 6-sided polygon, not a triangle.

Therefore, the mpv info line from the screenshot above should look like this:

Primaries: [0.680 0.320, 0.265 0.700, 0.150 0.050, 0.313 0.329] in bt.2020 Transfer: gamma2.2

Where the EDID primaries are simply rounded to three decimal places.

And mpv already does this correctly when displaying the mpv info for typical HDR video (photo from the nearby discussion I showed above):

Primaries: [0.680 0.320, 0.265 0.690, 0.150 0.060, 0.313 0.329] in bt.2020 Transfer: pq

Inside the square brackets are the pure, unclamped P3 primaries, even though the P3 colorspace triangle does NOT fit entirely within the BT.2020 colorspace triangle! This is correct; it should be exactly the same for a custom EDID colorspace. Don't clamp the primaries.

[kasper93]

First, I doubt it makes sense for this to be triggered in my case described above—where the Built-in (EDID) setting is enabled instead of an ICC profile. After all, this is a SDR display with gamma 2.2 (which mpv reads correctly). If it were an HDR display, it would make sense to encode the content into a BT.2020 container colorspace but restrict the colors by the custom EDID colorspace. But in the case of an SDR display, I would expect the data to be encoded by mpv directly into the custom EDID colorspace and sent to KWin in that form, so that KWin wouldn't have to perform any additional conversions. Of course, this would only work if KWin is capable of accepting input data encoded in a custom colorspace rather than a standardized one.

It does accept custom primaries as metadata. You still need encoding primaries, you cannot just encode free-form values into free-from buffer, because nothing will understand that, even with metadata. It's not how the things work. Find better LLM, because I will not spend time explain this against some hallucinating ideas of LLM.

Second, I disagree with "raw primaries are clamped inside the container primaries." Look, we can think of the container colorspace and the custom EDID colorspace as triangles on the x y plane, with their vertices defined by the respective primaries. The set of allowed colors can be thought of as the intersection of these two triangles. The situation is trivial when the custom colorspace triangle is entirely inside the container colorspace triangle—then we can simply clamp all RGB triplets by the custom colorspace. However, if the custom colorspace triangle does not fit entirely within the container colorspace triangle—which is exactly the case with my display's EDID primaries—then the intersection of the two triangles becomes a quadrilateral, a pentagon, or a hexagon. In this scenario, we would need to clamp all RGB triplets by both the custom colorspace and the container colorspace. But you CANNOT clamp the primaries themselves! Doing so makes no sense: clamped primaries still define only a triangle, whereas here the allowed color gamut is restricted by a 4-, 5-, or 6-sided polygon, not a triangle.

Technically, it's correct, after intersecting two triangles, you may end up with poligon with more sides. But this is completely irrelevant. The question you want to answer, is where is the RED primary. What value is the pure RED? It cannot be a line or multiple points, it's single point.

If your red primary is outside the bt.2020 encoding, than it cannot be used and we clamp to the intersection of the most saturated red that can be represented. Such that the colors that are outside the bt.2020 encoding are gamut mapped to fit inside. Specific case, which would happen when you don't perform gamut mapping and instead convert [1, 0, 0] display-p3 to bt.2020, you get [0.753833, 0.045744, -0.001210], see that there is negative value here? We cannot send negative value over the HDMI, would you agree? So regardless what we need to clip or gamut map it. And we do that by fitting the colorspace into the triangle that is defined inside encoding colorspace.

Inside the square brackets are the pure, unclamped P3 primaries, even though the P3 colorspace triangle does NOT fit entirely within the BT.2020 colorspace triangle! This is correct; it should be exactly the same for a custom EDID colorspace. Don't clamp the primaries.

It shouldn't be. I don't know what you want to achieve. You are looking at the numbers, which you don't understand, how they are produced and how they are used, and complain that they are incorrect? Could you step back a little and focus on actual issue? Please provide examples where produced image by mpv is incorrect, the image, not "numbers" and then we can discuss.

[FTKV]

Find better LLM, because I will not spend time explain this against some hallucinating ideas of LLM.

These arguments are not the product of any LLM generation.

It does accept custom primaries as metadata. You still need encoding primaries, you cannot just encode free-form values into free-from buffer, because nothing will understand that, even with metadata. It's not how the things work.

Are there obstacles to using custom primaries as encoding primaries? Can KWin accept custom primaries as encoding primaries?

The question you want to answer, is where is the RED primary. What value is the pure RED?

I am not looking for the answer to this question, provided that the red of the container colorspace and the red of the custom colorspace are already known.

If your red primary is outside the bt.2020 encoding, than it cannot be used and we clamp to the intersection of the most saturated red that can be represented. Such that the colors that are outside the bt.2020 encoding are gamut mapped to fit inside. Specific case, which would happen when you don't perform gamut mapping and instead convert [1, 0, 0] display-p3 to bt.2020, you get [0.753833, 0.045744, -0.001210], see that there is negative value here? We cannot send negative value over the HDMI, would you agree? So regardless what we need to clip or gamut map it.

Everything here is correct. You are essentially continuing my line of argument, as I have already mentioned twice that P3 does not fit entirely within BT.2020. There is no disagreement on this.

And we do that by fitting the colorspace into the triangle that is defined inside encoding colorspace.

But this is where my question lies—why define a new triangle with new "clamped" primaries if it leads to an additional, unnecessary restriction of the color gamut, which can be avoided? We can take an arbitrary triplet, convert it to the display colorspace, clamp it, then convert it to the container colorspace, and clamp it again. That's it—all pixels will now lie within the 4-, 5-, or 6-sided polygon intersection, and their values in the container colorspace will be bounded between 0 and 1. It doesn't seem overly complicated, does it?

For example, let the container colorspace be BT.2020 and the display colorspace be P3. Take a pixel with XYZ coordinates [0.2, 0.1, 0.00005], which lies outside both P3 and BT.2020. Converting it to P3 yields [0.40565564, 0.010369793, -0.0003992344]. Clamp it. Next, converting [0.40565564, 0.010369793, 0.] to BT.2020 yields [0.30776948, 0.028317024, -0.00030925802]. There is a negative value again, so we clamp it once more, resulting in the final pixel value in BT.2020: [0.30776948, 0.028317024, 0.].

If we repeat this process for all pixels, they will all lie on the xy plane within the 4-sided polygon that forms the intersection of the BT.2020 and P3 triangles.

To achieve all of this, there is NO need to define some new triangle that is completely inside the container colorspace. Otherwise, there might be points that are inside both the P3 and BT.2020 triangles, yet outside your new triangle, meaning you would clamp them even though there would be no issues rendering them. On the other hand, if you are referring to a specific, documented algorithm that explicitly requires defining a new triangle, please just provide its specification.

You are looking at the numbers, which you don't understand, how they are produced

I already know how those numbers are produced because I have reproduced them myself.

how they are used

However, the usefulness of using them is indeed questionable.

Could you step back a little and focus on actual issue? Please provide examples where produced image by mpv is incorrect, the image, not "numbers" and then we can discuss.

As far as I understand, mpv emphasizes not just good video playback, but mathematically accurate reproduction, and "numbers" do matter, would you agree?

[kasper93]

Are there obstacles to using custom primaries as encoding primaries? Can KWin accept custom primaries as encoding primaries?

Yes, standards.

I am not looking for the answer to this question, provided that the red of the container colorspace and the red of the custom colorspace are already known.

You, argued, that your colorspace is not triangle. So, this is the main question. How do you imagine this to work? We are working with RGB colorspace, it has 3 primaries, which can produce various colors by varying the intensity of them. Colorspace therefore is a triangle.

But this is where my question lies—why define a new triangle with new "clamped" primaries if it leads to an additional, unnecessary restriction of the color gamut, which can be avoided?

Again, how it can be avoided? You seem to be very sure that it can be avoided. I don't know how and when I asked you, you said "I am not looking for the answer to this question". So, I'm not sure how can we continue.

For example, let the container colorspace be BT.2020 and the display colorspace be P3. Take a pixel with XYZ coordinates [0.2, 0.1, 0.00005], which lies outside both P3 and BT.2020. Converting it to P3 yields [0.40565564, 0.010369793, -0.0003992344]. Clamp it. Next, converting [0.40565564, 0.010369793, 0.] to BT.2020 yields [0.30776948, 0.028317024, -0.00030925802]. There is a negative value again, so we clamp it once more, resulting in the final pixel value in BT.2020: [0.30776948, 0.028317024, 0.].

Clamping like that will skew hue, we have way more advanced gamut mapping than clamping. Additionally your proposal is clamping to P3 first which is smaller than bt.2020, and later clamping to bt.2020 again. Anyway, if you reverse order you would get inverse situation. But either way, what you end up in the end is a TRIANGLE that is your final colorspace. The fact you clamp twice is inconsequential, you arrive at the RGB colorspace which is triangle. Basically you can imagine that you do the clamping and get effective triangle, and this is what is shown in mpv stats page. I think you are close to understanding that, you realized that you need to clamp twice, which basically produces triangle that is intersection of those two.