output: fix automation for tearing + hw cursor + direct scanout

[ikalco]

Describe your PR, what does it fix/add?

related #9858 and #9890

this pr tries to fix tearing when its combined with other stuff (ds, hw cursor, maybe more? idk)
also draft cause seems to me some other old mechanisms (pre aq) may still be interfering with new ones

Is there anything you want to mention? (unchecked code, possible bugs, found problems, breaking compatibility, etc.)

Is it ready for merging, or does it need work?

draft

[UjinT34]

DS doesn't work with SW cursors. Might work with invisible SW cursors if the code allows it.
HW cursor movement breaks vrr on nvidia-drm (probably driver issue with atomic commit and cursor plane properties).
Locked cursors are handled differently with no_break_fs_vrr = 1. Might be the source of issues since tearing is usually needed for FPS games and they use locked cursors.

[abihf]

is there still any blocking for this PR? I've been using this patch for weeks and it's okay.

[ikalco]

is there still any blocking for this PR? I've been using this patch for weeks and it's okay.

I don't think its finished, gotta look over it still

[Honkazel]

@ikalco rebase?

[fxzzi]

rebase possible? ty ikalco

[Cloudperry]

I tried this on top of Hyprland dd33128 and it didn't seem to work properly. (This is the first time I have tested this PR.)

When I started a fullscreen game hyprctl monitors showed the same hex value on solitary and directScanoutTo. I guess this means that direct scanout was active. There were some graphical glitches and stuttering while it was active and I moved the mouse. When I went in game and the cursor was hidden, directScanoutTo was 0 and the stuttering/glitches stopped. When unhiding the cursor by going into a menu, directScanoutTo stayed as 0.

[fxzzi]

Yeah, just had some time to test too. Results came like this:

• There still seems to be some sort of inconsistency with DS, tearing, and hw cursor triggering at the same time. It's more apparent with animations disabled for some reason, but I often catch activelyTearing true but directScanoutTo 0.
• Moving hw cursor (cpu buffer) just kills DS for the rest of the time the game is fullscreened. Can sometimes be randomly recovered when flicking between workspaces or going between tiled / floating and fullscreen a few times.

[ikalco]

@fxzzi
can you tell me what app/game you use see if there's tearing cause I can't seem to get it? although its possible my gpu is too op hehe
also what options do you use for, misc:vrr, render:direct_scanout, general:allow_tearing, cursor:no_hardware_cursors

btw if you test the new commits out, I changed the activelyTearing and directScanoutTo to be simpler for the hyprctl monitors command

edit:
I just realized the weird behavior probably has something to do with nvidia so I'll try my old 1660 card sometime soon

[fxzzi]

I just realized the weird behavior probably has something to do with nvidia

likely, tho i've also noticed some strange behaviour on my friends AMD based PC (not tested this PR there yet). On certain games for him (Minecraft 1.8.9, Paradox games like crusader kings 3, hearts of iron 4, etc), if he changes workspace and back to the game, his fps is locked to a low number (like 30 or so) for about 10-15 seconds. Also happens when notifications show on screen etc.

I tested a few combinations with him:

# broken, fps locked for about 10 seconds
hyprctl keyword render:direct_scanout 1
hyprctl keyword general:allow_tearing 1
hyprctl keyword misc:vrr 2

# also broken
hyprctl keyword render:direct_scanout 0
hyprctl keyword general:allow_tearing 1
hyprctl keyword misc:vrr 2

# works (and so does vrr 0)
hyprctl keyword render:direct_scanout 0
hyprctl keyword general:allow_tearing 0
hyprctl keyword misc:vrr 2

I use the exact same setup with my nvidia GPU and don't experience this bug

can you tell me what app/game you use see if there's tearing cause I can't seem to get it?

I'm checking for tearing in a few games.

• Geometry Dash, which I run at double my monitor refresh rate experiences tearing as expected (340fps).
• Counter-Strike: Source, which I run at 3 below my refresh rate (167fps), theoretically shouldn't tear with vrr, tearing and ds all enabled at once, but still does. (i assume vrr should avoid tearing with tearing enabled, but without the latency increase of having tearing disabled)

EDIT: also maybe an nvidia related issue. Sometimes games feel stuttery and it magically fixes by going to a different TTY and back. idk y

cursor:no_hardware_cursors is false, using defaults for cursor stuff (cpu buffer on nvidia system, regular hw cursor on the amd sys)

vrr seems to be broken tho even if the cursor is invisible. Noticed this on ghost of tsushima. Controller works fine with VRR, but using my mouse causes my monitor OSD to stay at 170

[ikalco]

ok so the main issue is that the kernel doesn't really like anything changing, including the hw cursor position, while doing a tearing page flip
so my workaround is to use a seperate commit which shouldn't trigger any page flip in order to update the hw cursor position
however, the kernel only seems to like updating the hw cursor at vsync framerate, so moving the cursor limits the frames and could cause temporary stuttering

I'll try asking around about tearing and hw cursor, hopefully there are some better workarounds or a kernel mechanism I missed, but this is the best we got for now

---

theoretically shouldn't tear with vrr, tearing and ds all enabled at once, but still does. (i assume vrr should avoid tearing with tearing enabled, but without the latency increase of having tearing disabled)

not sure if this is the expected the behavior but it's up to the kernel since all we have are switches for tearing and vrr

also maybe an nvidia related issue. Sometimes games feel stuttery and it magically fixes by going to a different TTY and back. idk y

its probably related to the workaround limiting frames

vrr seems to be broken tho even if the cursor is invisible.

this is probably related to cursor:no_break_fs_vrr as UjinT34 suspected, but either way fixing vrr is kinda out of scope for this pr

[fxzzi]

thanks for the explanation, seems the kernel has more to do with ds tearing and vrr than I expected.

fixing vrr is kinda out of scope for this pr

yeah, I know, just thought I'd share anyway since it seems to be an issue in relation to them. my apologies :)

[Cloudperry]

I'm using an AMD gpu with Mesa git and Kernel 6.15.3. I have these tearing related settings:

general:allow_tearing 1
render:direct_scanout 1
misc:vrr 2

With the latest commits this PR doesn't cause stuttering or glitches, but still behaves a bit weirdly. When starting a game, tearing gets activated and scanout as well. However direct scanout keeps flipping between on and off unless I constantly move the mouse around. Also moving the mouse out of the game's monitor causes direct scanout to get stuck as off and it won't turn on when the cursor is back on the same monitor. Also it turns permanently off when the game hides the cursor (I was testing an FPS game).

[fxzzi]

tested your latest commits with hw cursor stuff disabled:

{
  programs.hyprland.settings = {
    cursor = {
      no_hardware_cursors = 1;
      min_refresh_rate = 32;
      no_break_fs_vrr = 1;
    };
  };
}

Still unable to get the two to activate at the same time reliably. The change in syntax for currentTearing and currentScanout also broke agsv1 for me, I think because of the space in the output:

(com.github.Aylur.ags:1582): Gjs-WARNING **: 02:36:20.202: JS ERROR: SyntaxError: JSON.parse: expected ',' or '}' after property value in object at line 28 column 29 of the JSON data @ resource:///com/github/Aylur/ags/service/hyprland.js:108:30
Hyprland@resource:///com/github/Aylur/ags/service/hyprland.js:108:30
@resource:///com/github/Aylur/ags/service/hyprland.js:334:25

edit: fwiw i think the false or true is useless, just show 0 / false if no window is actively being teared / ds'd, and then show the window id if there is. should avoid needing a space in the output

[fxzzi]

Comments from 2018 :woe:

Anyway, since this PR has gone slightly off the rails lol I would like to also point out #10452, but idk if it's trivial to add or not

[vaxerski]

I am fine with the rest

[ikalco]

@fxzzi test plz :)

edit:
also vaxry I made hw cursor always disable with tearing, since it just isn't possible for them to work together right now

[fxzzi]

@fxzzi test plz :)

Sorry 😔 won't be able to test for a few weeks as I'm on holiday

Hopefully someone else here is willing to test and we can get these changes into 0.50 :)

[DekoDX]

Tearing with this pr while using direct scanout appears to be working properly.

[vaxerski]

this should be bool at this point

[LionHeartP]

Rebase possible?

[UjinT34]

The title and description are kind of misleading. They leave an impression that DS, HW cursors and tearing will work simultaneously. While in fact it is "auto disable DS when tearing is active" + some edge cases. Auto disable DS part is already there since 0.50.0. Hard to tell what other edge cases need fixing. Probably everything could be handled my manual switching between two configs:

general {
  allow_tearing = 0
}
render {
  direct_scanout = 1
}
cursor {
  no_hardware_cursors = 0
}

and

general {
  allow_tearing = 1
}
render {
  direct_scanout = 0
}
cursor {
  no_hardware_cursors = 1
}

But the users expect all features to work at the same time with this config

general {
  allow_tearing = 1
}
render {
  direct_scanout = 1
}
cursor {
  no_hardware_cursors = 0
}

which is not going to happen any time soon since

// TODO: remove this when kernel allows tearing + hw cursor updated
// hw cursor can't be updated at the same time as tearing, so lock sw cursor
g_pPointerManager->lockSoftwareForMonitor(m_self.lock());

hyprctl monitors output with activelyTearing: True and non-zero directScanoutTo is probably a bug and shows some previous DS activation var value which wasn't cleared.

The only case when both DS and tearing can be active at the same time is when the cursor is hidden. But this case has no special handling iirc and DS will keep requiring HW cursors, tearing will keep requiring SW cursors even when no cursor is required.

[fxzzi]

you're probably tired of rebasing xd but now that #11228 is merged it might be a good time

[UjinT34]

Rebased this PR into a single commit ikalco#1
ikalco@596df10

Should be checked by @ikalco to verify that I didn't mess something up.

[fxzzi]

tested and works fine on my end.

[vaxerski]

will DRM accept direct scanout + tearing + hw cursors?

[ikalco]

will DRM accept direct scanout + tearing + hw cursors?

no 😔, it'll be a while until kernel devs implement it

[UjinT34]

What happens on AQ side after a failed test with a new format for DS? Looks like "set format - test - restore format" will set AQ_OUTPUT_STATE_FORMAT flag which will trigger an unneeded modeset . And it'll happen on every frame for buffers that can't be imported to KMS.
Probably the same thing happens on main... if it doesn't leave internal drm format state with an incorrect value.