NTsync (aka Winesync+Fastsync) - feedback topic

[Iglu47]

NTsync (title not approved) is more "Correctness" and "Robustness" alternative implementation of synchronization primitives in Wine from Zebediah Figura (the author of "Esync" and "Fsync").

This requires changes on the Wine side (usually patches are called - fastsync ) and corresponding changes on the kernel side (implemented as a kernel module - winesync. Unlike Fsync, the winesync functionality cannot be used anywhere except Wine).

More details about what problem NTsync solves and what is "implementation of synchronization primitives in Wine" can be found here: https://lkml.org/lkml/2021/1/17/312 and here: sync2022.pdf

Some plans and wishes for tests from @zfigura:

• Find or get good, convincing data vs server sync. Benchmarks of esync are hard to find and a lot of them are not exactly very convincing (on the order of 50 -> 54 FPS).

• Test ntsync with applications that break with fsync or esync:

  • [21142] Planet Zoo crashes when selecting a friend's avatar
  • [20155] Ubisoft launcher hangs on "looking for patches"
  • [17625] Yakuza 0 hangs on exit
  • [19072] Yakuza 0/Kiwami manual save fails
  • [21194] GTAV hangs or something. This issue: Grand Theft Auto V (271590) ValveSoftware/Proton#37
  • [20444] System Shock: Enhanced Edition hangs on exit

• Add this information to the document. I intend to submit this upstream along with the cover letter.

• Submit kernel patches upstream.

CALL FOR TESTS: anyone who can test games from the "breaks with esync/fsync" list, that would be quite appreciated. Note there's a lot more that break simply because of PulseEvent(), and we've had a couple tested already and they work so I'm not worried about those. All of the above break because of subtler timing problems and I'd like to see if they work with ntsync.

ALSO CALL FOR TESTS: I need good, convincing data comparing winesync vs server side sync. I don't have application names because in the 3-4 years since I wrote esync I've completely forgotten what the most important applications even are.

How to get NTsync working on TkG builds?

1. you need set _winesync="true" in .cfg for linux-tkg, build, install and loading on it
2. when you need set _use_fastsync="true" in .cfg for wine-tkg and build it
3. if you did everything correctly, then when you start the game in the output of the terminal or in the logs will appear
   wine: using fast synchronization.
4. to disable NTsync, set env var WINE_DISABLE_FAST_SYNC=1.

*Please don't post conflicts with other patches, versions of Wine, or other things related to building or installing kernel or Wine in THIS thread - this is a place intended for developers and other players to see other people's NTsync test results and determine if there are any significant performance deviations (both up and down) compared to wineserver-sync, esync, fsync. And any issues are saved or added compared to esync/fsync/wineserver-sync only.

[ms178]

I've got some numbers for you from the in-game-benchmark in Total War: Troy in 1080p / Ultra.

With NTSync:

Scene 1: 75,2 fps
Scene 2: 119,4 fps
Scene 3: 102 fps

WINE_DISABLE_FAST_SYNC=1:

Scene 1: 74,0 fps (-1,6 %)
Scene 2: 116,4 fps (-2,5 %)
Scene 3: 101,5 fps (-0,49 %)

Analysis: Scene 2 is the most CPU-bound of the three. As you see, there is quite a pronounced impact from the change.

My test system:

❯ inxi -GSC -xx
System:
  Host: klx99 Kernel: 6.1.5-3.2-cachyos-bore-lto arch: x86_64 bits: 64
    compiler: clang v: 16.0.0 Desktop: KDE Plasma v: 5.26.80 tk: Qt v: 5.15.8
    wm: kwin_x11 dm: SDDM Distro: CachyOS
CPU:
  Info: 18-core model: Intel Xeon E5-2696 v3 bits: 64 type: MT MCP
    arch: Haswell rev: 2 cache: L1: 1.1 MiB L2: 4.5 MiB L3: 45 MiB
  Speed (MHz): avg: 2312 high: 3790 min/max: 1200/2301 boost: enabled cores:
    1: 2301 2: 2301 3: 2301 4: 2301 5: 2301 6: 2301 7: 2301 8: 2301 9: 2301
    10: 2301 11: 2301 12: 2301 13: 1241 14: 2301 15: 2301 16: 2301 17: 2301
    18: 2301 19: 2301 20: 2301 21: 2301 22: 2301 23: 3790 24: 2301 25: 2301
    26: 2301 27: 2301 28: 2301 29: 2301 30: 2301 31: 2301 32: 2301 33: 2301
    34: 2301 35: 2301 36: 2301 bogomips: 165744
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Graphics:
  Device-1: AMD Vega 10 XL/XT [Radeon RX 56/64] vendor: Micro-Star MSI
    driver: amdgpu v: kernel arch: GCN-5 pcie: speed: 8 GT/s lanes: 16 ports:
    active: DP-3 empty: DP-1,DP-2,HDMI-A-1 bus-ID: 05:00.0 chip-ID: 1002:687f
  Display: x11 server: X.Org v: 21.1.99 with: Xwayland v: 22.1.7
    compositor: kwin_x11 driver: X: loaded: amdgpu unloaded: modesetting
    alternate: fbdev,vesa dri: radeonsi gpu: amdgpu display-ID: :0 screens: 1
  Screen-1: 0 s-res: 2560x1440 s-dpi: 96
  Monitor-1: DP-3 mapped: DisplayPort-2 model: HP X27q res: 2560x1440
    dpi: 109 diag: 685mm (27")
  API: OpenGL v: 4.6 Mesa 23.1.0-devel (git-267dd1f4d5) renderer: AMD
    Radeon RX Vega (vega10 LLVM 16.0.0 DRM 3.49 6.1.5-3.2-cachyos-bore-lto)
    direct render: Yes

[ms178]

Company of Heroes 2 - WQHD, Higher preset

With NTSync (2nd run):

Min: 56,00 fps
Avg: 79,84 fps
Max: 127,87 fps

WINE_DISABLE_FAST_SYNC=1:

Min: 56,99 fps (+1,7 %)
Avg: 80,44 fps (+0,75 %)
Max: 129,22 fps (+1,06 %)

Notes: There is a performance regression with NTSync in this older game. The first run with NTSync showed even worse numbers (77 fps avg) - but that might have been due to the shader compilation, hence I took the numbers from the second run which did show some improvements. I wouldn't read too much into this result as it is still in the range of normal deviation in this benchmark. Maybe still something worth to investigate?

[Tk-Glitch]

Maybe still something worth to investigate?

With only 2 runs it's hard to say if it's more than margin of error considering such minuscule differences. If the average on 10 runs leads to a similar gap, then that could be considered as an actual perf drop.

Now and that being said, if the WINE_DISABLE_FAST_SYNC=1 runs are with server-side sync, the difference should be much higher if you're CPU bound (>20% is quite common). If that's with fsync it's in line with what is to be expected from winesync, which is pretty much equal perf to fsync.

[Iglu47]

thanks for trying to look into this. I have access to the EGS-version of A Total War Saga: Troy so I did a couple of tests with "campaign_benchmark" at Ultra preset settings - there seems to be a difference between regular wineserver-sync and ntsync at least.

screenshots of the in-game "campaign_benchmark" result

server-side sync:

ntsync:

UPD. another run I made mangolog https://flightlessmango.com/games/26979/logs/3576

please make sure the terminal output or log show wine: using fast synchronization. before posting resault here (we want the test results to be valid)

[Tk-Glitch]

Those results are making much more sense @Iglu47 . Thanks.

[ms178]

I have no control over the build process, I simply used the proton-winesync binary packages ptr1337 provided in CachyOS for testers. That means by deduction that my numbers are highly likely just as valid as Iglu47's, as I've used the defaults even if they look different. Our hardware and software setup is very different after all.

I will leave more testing to others as I spend enough of my time with this experimental version that has other issues - even if Tk-Glitch considers these expected behavior. .

[Tk-Glitch]

@ms178 The proton build isn't enough for NTsync to work. Unless @ptr1337 added similar patches and handling to what we're doing in linux-tkg, you would need both winesync-dkms and winesync-udev-rule packages to begin with, and then either manually loading the module or adding it to your modules-load.d for it to be loaded on boot.

You didn't confirm you were seeing wine: using fast synchronization. in your wine output which would tell us if your results are indeed valid and everything was done correctly (either by you or the CachyOS maintainers), or not. You are talking with devs here, please try to understand we're expecting you to follow our guidelines if you participate in testing.

[ms178]

Please have mercy with me, I am not a programmer and just wanted to help you guys. :) Admittedly I had missed that crucial part of the process.
winesync-dkms / winesync-udev-rule were also installed. I assume ptr1337 handled all the neccesary patches, as he has seen NTSync working correctly with his binaries on other machines. The modules-load.d info is new to me though, I am deferring to prt1337 if that was handled automatically with his packages. If not, that could be a source for the deviation in the numbers. On the other hand, with a 1080Ti, the better result from Iglu47 with 140+ fps is also not too distant from mine with 119,4 fps and my Vega 56.

[FuzzyQuils]

Hello, your resident Overwatch player reporting in!

Soon as I have time I can get performance numbers vs server sync but I am glad to report Battle.net crashing wineserver in some cases with fsync doesn’t happen with winesync. (A reliable repro for the fsync issue is to also run Battle.net via Feral’s Gamemode with fsync enabled, such thing doesn’t happen using Winesync!)

[Iglu47]

[20155] Ubisoft launcher hangs on "looking for patches"

Still present with NTsync for me. Using "wineserver sync" helps as workaround.

[LethalManBoob]

i get hangs on ea's new game app

[csutcliff]

Battle.net client struggles with fsync (fails to launch 1/2 the time). So far it hasn't had an issue starting with winesync. In-game performance (WoW) appears to be the same or better than fsync but being an MMO it is fairly subjective.

[csutcliff]

Again not a perfect comparison as it's protonified vanilla wine vs actual proton with valve wine but Hogwarts Legacies is running better (10-15 fps at 4k ultra no RT) with fastsync than fsync and hasn't crashed on me yet which was frequently happening on load screens.

[ptr1337]

Again not a perfect comparison as it's protonified vanilla wine vs actual proton with valve wine but Hogwarts Legacies is running better (10-15 fps at 4k ultra no RT) with fastsync than fsync and hasn't crashed on me yet which was frequently happening on load screens.

You can also compile a fsync protonfied vanilla wine, then you would have a better comparison.
Also you can disable NTsync/Winesync with your current running wine/proton with:
WINE_DISABLE_FAST_SYNC=1

[LethalManBoob]

winesync tkg is gone from aur

[csutcliff]

winesync tkg is gone from aur

https://aur.archlinux.org/packages/winesync-dkms &
https://aur.archlinux.org/packages/winesync-udev-rule

works fine

[LethalManBoob]

There was a specific prebuild protonified wine version of tkg that supported winesync
edit: spoke to ptr 1337 this was due to tkg making changes that made the cachyos prebuilds unusable

[shmerl]

Is there some instruction how to build and test winesync? And what's the current status of it?

A side question, I recently encountered huge performance improvement in the Witcher 3 (DX12 mode) and Cyberpunk 2077 when Wine-esync is used. I wanted to compare it to Wine+fsync.

What is a good way to apply fsync patches to upstream Wine? I didn't find it in Wine-staging and it's not clear how to apply them from other sources.

[rlees85]

Another report for Battle.net working nicely with winesync. On both esync and fsync it would not launch half the time or cause wineserver to coredump sometimes taking the whole system with it.

I've been playing StarCraft 2 and Overwatch 2 - both used to work with fsync anyway - with winesync. No problems.

Performance in StarCraft 2 is better but it could be any number of reasons since wine-tkg (my custom build of)/wine-ge-custom are very different things.

[An-Eagle]

[20155] Ubisoft launcher hangs on "looking for patches"

Still present with NTsync for me. Using "wineserver sync" helps as workaround.

Not completely related to this thread, but the crypt32 dll isntalled through winetricks allows to get through this error in my experience, and has for a few other users I have helped in the past.

[shmerl]

Just a follow up. Any recommended way to use Winesync now over upstream kernel and wine? Various repos are quite outdated to apply things.

[FuzzyQuils]

Just a follow up. Any recommended way to use Winesync now over upstream kernel and wine? Various repos are quite outdated to apply things.

The winesync-dkms module + a recent build of wine-tkg with fastsync enabled in the config. That's generally what I do.

For Proton, it's a bit trickier, I've been working on a valve tree patchset for that.

[shmerl]

Recent build of wine-tkg with fastsync enabled in the config

I tried playing around with that, but couldn't figure out how to apply only that on top of upstream Wine. wine-tgk has a ton of stuff and is a bit cumbersome to configure. No matter what I tried, it seemed to pull some extra stuff that I didn't want.

It would be nice to have something like wine-staging patchset for it that is easy to apply to upstream Wine.

And which repo has an up to date dkms module for winesync?

[shmerl]

What's the current state of this in general? I.e. is it close to getting upstream or more data is needed? Are there any blockers? Not sure where / how to check the status, so asking here.

[Iglu47]

@shmerl the status has not changed.
Important Information: Reports are needed to ensure that NTsync/Winesync does not introduce any new issues compared to Esync.

[ptr1337]

@shmerl the status has not changed. Important Information: Reports are needed to ensure that NTsync/Winesync does not introduce any new issues compared to Esync.

Generally a rebase against proton 8 would be very interesting for many users, i think.

[Iglu47]

Generally a rebase against proton 8 would be very interesting for many users, i think.

porting to wine-valve tree and availability of wenesync module. I'll try to find the time and do something about it

[csutcliff]

Is there anyone working on or planning on re-basing against the current wine master since it broke?

[daktras420]

thank you Iglu47 i was planning on doing wine build this week using your patchset and the wine-valve tree as well as proton for steam with the ntsync patches if its doable what reversion or current tree version are your patches compatible with ei. wine-protonified, or wine-protontified bleeding edge, same for steams proton which build is most compatible? for now since i have dxvk gasync installed via aur with arch will try it using system dxvk and see if it all works till and if the developer of the new dxvk patch files work is included into the frogging family of patches.

[shmerl]

What exactly are the issues?

[ptr1337]

What exactly are the issues?

Actually I didn't tested it on my own, but our users are complaining that with the latest patchset from lkml, it just stucks when starting anything with a ntsync patched wine.
Downgrading to an older kernel does fix it - so likely the latest patchset is broken.

That patch is currently used and recently updated by Zen,TKG and CachyOS

[BarackOBusiness]

Still seems to have the same problem, can't generate the default prefix or launch anything because of it, should I also downgrade the patch used for wine or something?

[Artewar67]

Still seems to have the same problem, can't generate the default prefix or launch anything because of it, should I also downgrade the patch used for wine or something?

You don't have winesync-dkms or ntsync-dkms installed?

[GrabbenD]

This configuration works ridiculously good
(*apart from having to raise ulimit -Hn to 214748364 due to memory leak in NTSync5)

• Distro: Arch Linux
• DE: Sway 1.9 (see my settings here)
• DKMS: NTSync5 https://aur.archlinux.org/packages/ntsync-dkms
• Kernel: Xanmod-Edge 6.8.5 https://aur.archlinux.org/packages/linux-xanmod-edge (games performs identically to LINUX-TKG https://github.com/Frogging-Family/linux-tkg)
• WINE GUI: Bottles https://aur.archlinux.org/packages/bottles
• WINE PREFIX: TKG 9.6 https://github.com/Frogging-Family/wine-tkg-git/tree/master/wine-tkg-git
  Default settings (wine-tkg-git/customization.cfg) apart from:
  • _LOCAL_PRESET="none" , _use_staging="false" , _use_fastsync="false" , _use_ntsync="true" , _use_esync="false" , _use_fsync="false" , _wayland_driver="true" , _proton_fs_hack="true" , _win10_default="true"

Been using it for weeks on end without any crashes. Zero. Works perfectly with VRR. Not going to update for a while..

Anecdotally, I never managed to get NTSync to work properly with CachyOS's kernel 😞 .. When it was detected, VRR wasn't. Probably client-kernel version mismatch (this was a couple of weeks ago).

[loathingKernel]

Still seems to have the same problem, can't generate the default prefix or launch anything because of it, should I also downgrade the patch used for wine or something?

You don't have winesync-dkms or ntsync-dkms installed?

winesync doesn't conflict with ntsync any more since ntsync RFC v2. The reason they used to conflict was because of the device number winesync reserves and ntsync used to reserve https://lore.kernel.org/lkml/20240124004028.16826-3-zfigura@codeweavers.com/

I have experienced similar issues with ntsync5 v4 kernel built-in on linux-zen-6.8.9 from Arch, which is the same one as the one in your repository zen-kernel - your repo

The wine I used for testing was patched with ntsync5 v4 from https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync5. You can find the extracted patch here

The wine build mentioned above (also available in my releases if anyone wants to validate it) worked with the ntsync-dkms module (v3) on linux-zen-6.7.9 but it did fail to create a prefix and generally work with the built-in ntsync from linux-zen-6.8.9.

This is also the reason I have postponed updating the AUR package for ntsync-dkms

[Artewar67]

Well, it looks like I'm the only one who has this version working. For now I will only work on ntsync for proton and wine

[Artewar67]

Before that, we used to make a patch from this repository: https://repo.or.cz/linux/zf.git/shortlog/refs/heads/ntsync5
It has more differences than the one from lkml. Here you can compare: CachyOS/kernel-patches@14bb253#diff-7c7d2ce599a28db9f9acefbe5de3953f6cb9847ea2c7294c5402b048e66591ee (Update ntsync to v4)

[loathingKernel]

The issue is with ntsync-header from AUR. Since linux-api-headers doesn't have ntsync.h yet, and wine looks for the header in /usr/include/linux/ntsync.h (the kernel provides it under lib/modules/<kernel>/build/include/uapi/linux/ntsync.h but wine doesn't look for it there), the v3 version from AUR was used to compile wine which is incompatible with the v4 version of the module.

Thus the AUR package has been updated and you will need to rebuild wine for it to work properly. Previously compiled versions will not work.

[Artewar67]

I thought the problem was more complicated than that. I created just a symbolic link in /usr/include/linux/ to /usr/lib/modules/"kernel"/build/include/uapi/linux/ntsync.h and forgot that just wine except linux/ntsync no (I don't remember exactly what the message was) in the build log will not report anything else about the problem

[Nanotwerp]

The issue is with ntsync-header from AUR. Since linux-api-headers doesn't have ntsync.h yet, and wine looks for the header in /usr/include/linux/ntsync.h (the kernel provides it under lib/modules/<kernel>/build/include/uapi/linux/ntsync.h but wine doesn't look for it there), the v3 version from AUR was used to compile wine which is incompatible with the v4 version of the module.

Thus the AUR package has been updated and you will need to rebuild wine for it to work properly. Previously compiled versions will not work.

Ah, you're right! I'm on NixOS, but the same issue applies. I overrode my kernel to be one patched with ntsync5, and the ntsync module successfully compiles, but I didn't realize that the ntsync.h header isn't being propagated to Wine. After looking at my build logs for Wine, I noticed it saying something like what @Artewar67 said (checking for linux/ntsync.h... no) indicating that ntsync.h wasn't found inside the kernel headers directory. For NixOS in particular, this is because kernel headers are decoupled from the default kernel package.

After overriding Wine to use the same kernel headers as my overridden kernel uses, it successfully finds ntsync.h. After loading the module with sudo modprobe ntsync --first-time, Wine successfully creates a prefix and runs programs! So, as @loathingKernel said, it's simply a header issue, not intrinsically because of the update from ntsync5-v3 to ntsync5-v4 itself. This means the patch on @ptr1337's kernel-patches repo for CachyOS can be updated again.

[bunkbail]

I built wine-tkg as per comment and now getting the error Cannot open /dev/ntsync: Permission denied. I'm using Xanmod-edge patchset on Debian as it includes ntsync patches and all.

[Nanotwerp]

I built wine-tkg as per comment and now getting the error Cannot open /dev/ntsync: Permission denied. I'm using Xanmod-edge patchset on Debian as it includes ntsync patches and all.

That's a udev rules issue! By default, /dev/ntsync has 0600 permissions, meaning only the root user can read and write to it. You can add a rule that gives it 0644 permissions, so that other users can open it.

[bunkbail]

That's a udev rules issue! By default, /dev/ntsync has 0600 permissions, meaning only the root user can read and write to it. You can add a rule that gives it 0644 permissions, so that other users can open it.

Thanks for the pointer! I ran echo 'KERNEL=="ntsync", MODE="0644"' | sudo tee -a /etc/udev/rules.d/ntsync.rules ; sudo udevadm trigger and it's working fine now!

wine: using fast synchronization.

[MCPO-Spartan-117]

Some people seem to think v4 is causing issues, however unless the change notes from lkml are wrong the only things changed from v3 to v4 is

1. The first 3 patches are included in 6.10 and no longer are being summited via the patch list
2. Selftests have a extra variable
3. The primitives have some more documentation

So despite the version bump, v4 and v3 are the same, functionally.

https://lore.kernel.org/lkml/20240416010837.333694-1-zfigura@codeweavers.com/

== Previous versions ==

Changes from v3:

* Add .gitignore and use KHDR_INCLUDES in selftest build files, per Muhammad
  Usama Anjum.

* Try to explain why we are rolling our own primitives a little better, per Greg
  Kroah-Hartman.

Also if the header still isn't installed by the package you can put it in /usr/local/include/linux/ntsync.h and it'll be seen by anything searching for headers while staying out of the system's directories until the packages or kernel headers update to include it properly.

[loathingKernel]

So despite the version bump, v4 and v3 are the same, functionally.

The difference was in the ntsync.h header and mostly my fault. Due to my tracking of the git repo instead of the lkml, after a rebase I version-bumped the kernel module to what I mistakenly thought was v3, instead it was a rebased v2. Between v2 and v3 this commit likely got squashed leading to a difference in the ntsync_wait_args struct, which in turn caused Wine compiled using the ntsync-header package from AUR to not work with later versions of the module. This didn't cause issues while the kernel module and the header were in the same version, so it went unnoticed. Once kernels started shipping the module, the incompatibility became apparent.

[MCPO-Spartan-117]

It seems the NTSYNC kernel patches have been post-poned to 6.11.
https://www.phoronix.com/news/Linux-6.10-NTSYNC-Broken

[wsy2220]

With ntsync on, I'm getting occasional frametime spike (100+ms) in GTAV driving around the city. Average fps improvement is significant though (160ish->180ish).

Vanilla wine has lower but stable framerate.

kernel: upstream 6.10.2 + ntsync patch from linux-tkg
wine: upstream 9.15 + dxvk 2.4 + ntsync patch from wine-tkg
hw: 5800x3d + RTX3090

Edit: liquorix kernel works great. I guesss it's a scheduler problem?

[OneOfOne]

With ntsync, battle.net's background login service keeps crashing non-stop.

[archer920]

Sorry - total noob here trying to get this to build on arch

I built linux-tkg with _winesync="true" in the configuration file but when I build wine-tkg I keep getting this error

==> ERROR: fatal ==> ERROR: Winesync header was not found. Build is forcibly preventd. ==> ERROR: Please use linux-tkg build with _winesync="true"...

[ptr1337]

Sorry - total noob here trying to get this to build on arch

I built linux-tkg with _winesync="true" in the configuration file but when I build wine-tkg I keep getting this error

==> ERROR: fatal ==> ERROR: Winesync header was not found. Build is forcibly preventd. ==> ERROR: Please use linux-tkg build with _winesync="true"...

Its better, that you choose to use "ntsync" instead of winesync.
Besides, that you want to install "ntsync-dkms", "ntsync-common" and "ntsync-headers" from the AUR

[archer920]

@ptr1337 - Thank you! I was able to get it to bulid but only with proton wine. The other ones resulted in compiler errors.

[OneOfOne]

So the issue was wine 9.16, with 9.15 battle.net starts, however the UI is black.