cm: handle CM for SDR content with cm=hdr, cm_sdr_eotf=2

[njdom24]

Describe your PR, what does it fix/add?

Primarily, stops colors from getting blown out when viewing some SDR content with cm=hdr.

Some SDR surfaces were both skipping color management and getting DS in HDR mode (cm=hdr, not Auto-HDR) when they shouldn't have been. This PR makes the conditions more strict by checking for SDR<->HDR mismatch, and differentiating being in HDR due to Auto-HDR, or manually using cm=hdr(edid).

Fixing this also brought to light how cm_sdr_eotf=2 would cause DS blockage because rendering sRGB EOTF as Gamma 2.2 on a Gamma 2.2 monitor still compared them in canNoShaderCM() as srgb != gamma22, so I threw in a check for that.

Resolves #12104

Is there anything you want to mention? (unchecked code, possible bugs, found problems, breaking compatibility, etc.)

Tested cm=hdr with VLC and MPV, and gamescope: No blown-out colors
Tested cm_sdr_eotf=2 with vkcube: color management not listed in directScanoutBlockedBy

Is it ready for merging, or does it need work?

Ready for merging

[vaxerski]

@UjinT34

[UjinT34]

cm_fs_passthrough = 1 is meant to solve this (should switch back to sdr).
Do you really want to keep monitor in HDR mode and have less than ideal SDR -> HDR without DS while the content is SDR and is compatible with DS? Doesn't make much sense. It only avoids a black screen during sdr/hdr switch.

Probably should be merged as is if it doesn't break other setups.

[njdom24]

@UjinT34 I do want to make the always-on HDR experience as good as possible, like it was from my time using KDE. I like the Auto-HDR feature, and I make use of it a lot, but sometimes I get tired of waiting for a modeset when I toggle fullscreen.

My monitor is a Mini-LED, and its local dimming feature is funky in SDR mode. Consuming media in general looks better with HDR on as long as the SDR -> HDR conversion is okay, which I did #12094 to address.

[LionHeartP]

My monitor is a Mini-LED, and its local dimming feature is funky in SDR mode. Consuming media in general looks better with HDR on as long as the SDR -> HDR conversion is okay, which I did #12094 to address.

I'm on the same boat, miniled with long modeset times, that's why I use cm=edid (or alternatively cm=wide) to avoid the modeset.

When autoHDR kicks in, the modeset is instant since wide is already in the correct mode, at least on my case. Cba to calibrate cm=hdr since it looks horrendous on my monitor ootb.

[njdom24]

Finally got around to testing cm_fs_passthrough = 1. Both on main and with this branch, the monitor stays in HDR mode and still blows out the colors. Only difference is that with this branch, the colors aren't blown out with cm_fs_passthrough = 2, because DS content falls through and blows out the colors anyway. non_shader_cm = 0 doesn't help without this PR.

So this branch is an improvement (fixes cm_fs_passthrough = 2), but maybe there's a problem with cm_fs_passthrough = 1 on my setup.

[njdom24]

Rebased and acknowledged different cm_fs_passthrough settings

[s0mebodyhelpme]

It seems this fixes the issue where cm_fs_passthrough = 1 would oversaturate and darken windows made to be fullscreen regardless of direct_scanout's setting.