Discoveries in the Course of Pushing Limits

[cerebrate]

To expand on that title, I'm Alistair Young ( @cerebrate ), and have spent rather a lot of time since WSL originally shipped pushing it to do things which, shall we say, weren't exactly in the original design specs. (The most prominent fruit of this is genie, a utility which lets you run systemd and all of its associated services inside WSL, officially supported or no.) My habits haven't changed since the advent of WSLg, so this thread is going to serve me as a place to share the things I discover in the course of pushing on WSLg with my fellow hackers and lunatics.

So, let's begin with this one:

init-+-dbus-daemon
     |-init(mnt,pid,uts)-+-init---unshare(mnt)---systemd(pid)-+-atd
     |                   |                                    |-automount---3*[{automount}]
     |                   |                                    |-colord(mnt)---2*[{colord}]
     |                   |                                    |-cupsd---{cupsd}
     |                   |                                    |-daemon.start(mnt)---lxd-+-dnsmasq
     |                   |                                    |                         `-18*[{lxd}]
     |                   |                                    |-dbus-daemon
     |                   |                                    |-lxcfs(mnt)---2*[{lxcfs}]
     |                   |                                    |-msmtpd(mnt)
     |                   |                                    |-polkitd---2*[{polkitd}]
     |                   |                                    |-rpcbind
     |                   |                                    |-rsyslogd---3*[{rsyslogd}]
     |                   |                                    |-snapd---14*[{snapd}]
     |                   |                                    |-systemd-+-(sd-pam)
     |                   |                                    |         |-dbus-daemon
     |                   |                                    |         |-emacs---{emacs}
     |                   |                                    |         |-gnome-keyring-d---3*[{gnome-keyring-d}]
     |                   |                                    |         |-gvfs-afc-volume---3*[{gvfs-afc-volume}]
     |                   |                                    |         |-gvfs-goa-volume---2*[{gvfs-goa-volume}]
     |                   |                                    |         |-gvfs-gphoto2-vo---2*[{gvfs-gphoto2-vo}]
     |                   |                                    |         |-gvfs-mtp-volume---2*[{gvfs-mtp-volume}]
     |                   |                                    |         |-gvfs-udisks2-vo---3*[{gvfs-udisks2-vo}]
     |                   |                                    |         |-gvfsd---2*[{gvfsd}]
     |                   |                                    |         |-gvfsd-fuse---5*[{gvfsd-fuse}]
     |                   |                                    |         |-kbfsfuse---14*[{kbfsfuse}]
     |                   |                                    |         |-keybase---15*[{keybase}]
     |                   |                                    |         |-keybase-redirec---10*[{keybase-redirec}]
     |                   |                                    |         |-tmux: server---zsh
     |                   |                                    |         |-tracker-miner-f---6*[{tracker-miner-f}]
     |                   |                                    |         |-zsh---gitstatusd-linu---8*[{gitstatusd-linu}]
     |                   |                                    |         |-zsh---zsh
     |                   |                                    |         `-zsh
     |                   |                                    |-systemd-journal
     |                   |                                    |-systemd-logind(mnt,uts)
     |                   |                                    |-systemd-machine(uts)
     |                   |                                    |-systemd-network(mnt)
     |                   |                                    |-systemd-resolve(mnt)
     |                   |                                    |-systemd-socket-
     |                   |                                    |-systemd-timesyn(mnt,uts)---{systemd-timesyn}
     |                   |                                    |-systemd-udevd(mnt,uts)
     |                   |                                    |-tor(mnt)
     |                   |                                    |-udisksd---4*[{udisksd}]
     |                   |                                    |-unattended-upgr---{unattended-upgr}
     |                   |                                    |-upowerd(mnt,user)---2*[{upowerd}]
     |                   |                                    |-watchdog
     |                   |                                    `-zsh-+-(sd-pam)
     |                   |                                          `-tmux: client
     |                   |-init---init---genie-+-machinectl
     |                   |                     `-8*[{genie}]
     |                   `-2*[{init}]
     |-init---WSLGd-+-dbus-daemon
     |              |-mstsc.exe
     |              |-pulseaudio---2*[{pulseaudio}]
     |              `-weston-+-Xwayland
     |                       `-14*[{weston}]
     |-init---init---bash---pstree
     `-{init}

What you are looking at here is a process tree taken from inside the WSLg system distribution on my machine, while it is running a single Debian distribution, while it is in turn running genie and systemd. As you can see, namespace transitions are included next to those processes which use them, so you can see what's going on.

If you've been following along with WSL hackery for long, you know that every distro running inside WSL essentially runs inside its own container-style namespace(s), sharing the single kernel instance and VM. This is why, for example, kernel threads don't show up in a ps aux, and certain non-namespaced features are shared between distros so that, for example, kernel modules loaded in one distro show up when you lsmod in a different distro.

WSLg appears to complicate this a bit. In the leftmost major column there, you have the processes of the system distribution, but what you will notice, the second major column, is that the processes of the user distribution, my Debian distro, are child processes of the system distro's init. That is to say, the user distro now runs as a mount, pid, and UTS namespace within the system distro.

Under that you can see genie's work (the line containing unshare and systemd) which creates another mount and pid namespace in order to create an environment in which systemd can own pid 1. While the details of the implementation vary, this much is common to all the hacks which run systemd under WSL. And then beneath it, in turn, you can see all the systemd services (including the user systemd) which it is happy to manage for you.

Below it, you can see genie invoking machinectl, which is its way to get systemd-containerd to handle all the work of setting up a login session properly for it.

So, yes, the best way I've found to run everything is in a container inside a container inside a container...

COMING SOON: I talk about getting WSLg to work in this arrangement, dbus and its travails, and the bits of Gnome that you can see working there...

[spronovo]

Nice write up. If you are curious as to why we decided to run the system distro as a sort of parent of it's associated user distro, it was two fold. First to allow us to share IPC namespace so we can use shared memory seamlessly (for example for X11 MIT-SHM protocol)... also because the system distro sometime needs to attach to the pid and mountspace of the user distro it's paired with (to find .desktop file, load icon files, etc...). That seemed the nicest and cleanest way to achieve this transparently.

[cerebrate]

Always curious about that sort of thing, I am indeed. 😀 Thanks!

[cerebrate]

On Making WSLg and Systemd Play Nicely Together

(in genie, anyway; also, credit for a lot of this should go to @diddledan , who figured it out first for his systemd solution)

There are three things genie does to play nicely with WSLg:

1. Copy the environment variables.

When systemd is started up inside its nice new pid namespace, it doesn't inherit the environment of the user distro namespace; which is what you want, really. Starting a systemd-as-pid-1 is essentially booting a Linux system, so it's starting from a clean slate. For this reason, genie already included mechanisms to clone certain environment variables (WSL_DISTRO_NAME,WSL_INTEROP,WSLENV) and the outside path into the "bottle" to keep essential functions working, so it was as simple as adding the WSLg environment variables (DISPLAY,WAYLAND_DISPLAY,PULSE_SERVER) to the default list to be cloned.

2. Map the /tmp/.X11-unix/X0 socket

Systemd, in its default configuration, does a lot of things to initialize the system. Unfortunately for us, one of them is to clean up the /tmp directory, via a unit named systemd-tmpfiles-setup, which blows away the link WSLg has established to the XWayland socket at /mnt/wslg/.X11-unix/X0. (Ordinarily, this is desirable - on a regular Linux system, it would be cleaning up detritus left over from previous boots.) This is a very important link, since X will look for the Unix sockets for local displays in /tmp/.X11-unix and nowhere else.

Now, we could just disable that unit, or edit the configuration files for systemd-tmpfiles not to touch that folder, but ideally we would like to make this work without having to modify bits of existing distros, especially bits that are fairly deeply embedded in their startup sequence. So instead, we make and install a couple of systemd units of our own, taking advantage of systemd-socket-proxyd:

wslg-xwayland.socket

[Unit]
Requires=systemd-tmpfiles-setup.service
After=systemd-tmpfiles-setup.service
ConditionVirtualization=wsl
ConditionPathExists=/mnt/wslg/.X11-unix/X0

[Socket]
ListenStream=/tmp/.X11-unix/X0

[Install]
WantedBy=sockets.target

And wslg-xwayland.service

[Unit]
Requires=wslg-xwayland.socket
After=wslg-xwayland.socket
ConditionVirtualization=wsl
ConditionPathExists=/mnt/wslg/.X11-unix/X0

[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd /mnt/wslg/.X11-unix/X0

which arranges for systemd itself to proxy connections to the X server to the proper location for WSLg.

3. Mount the runtime directory in the expected place

WSLg locates the various other sockets and files used to access parts of it under /mnt/wslg/runtime-dir. For most purposes, this is fine, since the environment variables inform clients of that, and everything works. But then you get to things like snaps, or certain apparmor profiles, or other things that expect the user runtime directory to exist under /usr/run/{UID} and will break if it doesn't find it there.

To deal with this one, we need the systemd unit that sets up the user runtime directories to instead use the WSLg runtime directory, but only for the primary user . That criterion is because things expect the user runtime directory to be owned by the user it is for, which (for /mnt/wslg/runtime-dir) is UID 1000 (wslg, inside the system distro); conveniently, though, user distros tend to set up the first user created as UID 1000 also, and the /mnt/wslg/runtime-dir will be perceived as owned by this user from inside the user distro.

To achieve this, we add an override file to the user-runtime-dir@.service, as follows:

user-runtime-dir@.service.d/override.conf

[Service]
ExecStart=/usr/libexec/genie/map-user-runtime-dir.sh %i
ExecStop=/usr/libexec/genie/unmap-user-runtime-dir.sh %i

running one of these two scripts:

map-user-runtime-dir.sh

#!/bin/sh
if [ ! -d /mnt/wslg/runtime-dir ]
then
  # WSLg is not present, so do nothing over-and-above previous
  exit
fi

WUID=$(stat -c "%u" /mnt/wslg/runtime-dir)

if [ $1 -eq $WUID ]
then
  # We are the WSLg user, so map the runtime-dir
  mount --bind /mnt/wslg/runtime-dir /run/user/$1
  exit
fi

map-user-runtime-dir.sh

#!/bin/sh
if [ ! -d /mnt/wslg/runtime-dir ]
then
  # WSLg is not present, so do nothing over-and-above previous
  exit
fi

WUID=$(stat -c "%u" /mnt/wslg/runtime-dir)

if [ $1 -eq $WUID ]
then
  # We are the WSLg user, so unmap the runtime-dir
  umount /run/user/$1
  exit
fi

These don't replace the default user-runtime-dir@.service; it still creates the user runtime directory under /run/user. What they do do is check if WSLg is present on the machine, and then check if the user they are currently creating a runtime directory for is the user who owns the /mnt/wslg/runtime-dir directory (i.e., has UID 1000) If so, they bind mount the WSLg runtime directory over the /run/user/{UID} directory, and then everything is in the right place for the purposes of those apps with expectations about these things.

Note: a side-effect of this is that other applications that use the user runtime directory will be creating their sockets, files, etc., under the WSLg runtime directory - for example, my /mnt/wslg/runtime-dir looks like this, with the various other things I run:

total 0
srw-rw-rw- 1 avatar avatar   0 May  2 11:11 bus=
drwx------ 3 avatar users   60 May  2 11:10 dbus-1/
drwx------ 2 avatar avatar  60 May  2 11:11 dconf/
drwx------ 2 avatar avatar  60 May  2 11:13 emacs/
drwx------ 2 avatar avatar 140 May  2 11:11 gnupg/
dr-x------ 2 avatar avatar   0 May  2 11:11 gvfs/
drwx------ 3 avatar avatar 140 May  2 11:13 keybase/
drwx------ 2 avatar avatar 100 May  2 11:14 keyring/
srw-rw-rw- 1 avatar avatar   0 May  2 11:11 pk-debconf-socket=
drwxr-xr-x 2 avatar avatar  60 May  2 11:11 podman/
drwx------ 2 avatar users   80 May  2 11:10 pulse/
srw-rw-rw- 1 avatar avatar   0 May  2 11:11 snapd-session-agent.socket=
drwxr-xr-x 6 avatar avatar 160 May  2 11:14 systemd/
srwxrwxrwx 1 avatar users    0 May  2 11:10 wayland-0=
-rw-rw---- 1 avatar users    0 May  2 11:10 wayland-0.lock

I've observed no adverse or indeed non-adverse side-effects from this (and it is, after all, a tmpfs), but it's probably worth noting for reference.

I hope this helps you if you're trying to put together your own systemd/WSLg hack. Of course, you could just use mine... 😁

[UGatgithub]

I am not a programmer and mainly use Linux for fun, so the Windows Subsystem for Linux unfortunately wasn’t very usable for me before WSLg arrived.

When I read about WSLg I knew this is something I want to try out now. It took me a while to understand the basic architecture behind but once I understood I thought that this is not only a possibility to avoid a Windows/Ubuntu double boot installation on my new PC but also to try out and run different linux apps and even different distros under wsl2. It is really great to see that quite some GUI apps already work under WSLg now.

Today I found and directly installed the GWSL Windows App on my main (non Insider) Windows Pro installation which works well there but not yet with the Windows Insider WSLg enabled Ubuntu Community Preview that I installed as Windows Insider on another SSD on the same PC.

Looking forward to the further development of this interesting topic!

[cerebrate]

The Dbus Status Quo - An Early Look

If you glance again at the process tree up at the start of this thread, you will see that I have four dbus-daemon instances running, between the system and user distros. Let's go through them one by one.

First, there is a dbus-daemon running as a direct child of init in the system distro. Command line:

dbus-daemon --syslog --fork --print-pid 5 --print-address 7 --session

Apparently a session bus. I have no idea what this one is doing yet, because I can't connect to it even with the help of lsof; dbus-monitor --session or dbus-send --session won't connect to it.

Second, there is a dbus-daemon running as a child of WSLg in the system distro. Command line:

dbus-daemon --syslog --nofork --nopidfile --system

This would appear to be a conventional system bus. Looking at the names it knows about --

root@pallas [ ~ ]# dbus-send --system --dest=org.freedesktop.DBus --type=method_call --print-reply /org/freedesktop/DBus org.freedesktop.DBus.ListNames

method return time=1619983268.498988 sender=org.freedesktop.DBus -> destination=:1.19 serial=3 reply_serial=2
   array [
      string "org.freedesktop.DBus"
      string ":1.19"
   ]

If my understanding is correct - and I freely admit that my understanding of DBus and associated matters is, at the moment, quite limited - there's nothing actually registered with this one. org.freedesktop.DBus is the service which represents the dbus-daemon itself.

That sums up the dbus-daemons found in the system distro.

Now, as for the user distro, the only dbus-daemons in there are the ones created by my systemd (using the genie hack), both system and session. I won't bother running through all the details of those when you just as easily can get them from your own distro, but the important point is that everything that uses DBus in the user session is talking to them:

❯ dbus-send --system --dest=org.freedesktop.DBus --type=method_call --print-reply /org/freedesktop/DBus org.freedesktop.DBus.ListNames
method return time=1619983865.193450 sender=org.freedesktop.DBus -> destination=:1.33 serial=3 reply_serial=2
   array [
      string "org.freedesktop.DBus"
      string "org.freedesktop.login1"
      string ":1.7"
      string ":1.8"
      string "org.freedesktop.timesync1"
      string "org.freedesktop.machine1"
      string "org.freedesktop.ColorManager"
      string "org.freedesktop.systemd1"
      string ":1.20"
      string "org.freedesktop.PolicyKit1"
      string ":1.21"
      string ":1.10"
      string ":1.33"
      string ":1.11"
      string "org.freedesktop.UPower"
      string ":1.12"
      string "org.freedesktop.UDisks2"
      string ":1.13"
      string ":1.0"
      string ":1.1"
      string ":1.15"
      string ":1.2"
      string "org.freedesktop.network1"
      string ":1.3"
      string ":1.17"
      string ":1.4"
      string "org.freedesktop.resolve1"
      string ":1.18"
      string ":1.19"
      string ":1.6"
   ]

❯ dbus-send --session --dest=org.freedesktop.DBus --type=method_call --print-reply /org/freedesktop/DBus org.freedesktop.DBus.ListNames
method return time=1619984147.525433 sender=org.freedesktop.DBus -> destination=:1.39 serial=3 reply_serial=2
   array [
      string "org.freedesktop.DBus"
      string ":1.7"
      string ":1.8"
      string ":1.9"
      string "org.gtk.vfs.Metadata"
      string "org.a11y.Bus"
      string ":1.30"
      string "org.freedesktop.systemd1"
      string "org.gtk.vfs.UDisks2VolumeMonitor"
      string ":1.31"
      string ":1.32"
      string "org.gnome.keyring"
      string ":1.11"
      string "org.gtk.vfs.AfcVolumeMonitor"
      string ":1.0"
      string "org.gtk.vfs.Daemon"
      string ":1.36"
      string "org.freedesktop.secrets"
      string "org.freedesktop.Tracker1.Miner.Files"
      string "org.gtk.vfs.MTPVolumeMonitor"
      string ":1.3"
      string "org.gtk.vfs.GoaVolumeMonitor"
      string ":1.39"
      string ":1.17"
      string ":1.4"
      string "org.gtk.vfs.GPhoto2VolumeMonitor"
      string ":1.5"
      string ":1.6"
   ]

As such, I think - if one is willing to use a systemd hack or other method to get both dbus-daemons running in your user session - I don't think DBus is a locus of problems, at this time, for getting Gnome or other DBus-using applications to work under WSL/WSLg, or at least I have not found such problems yet.

Anyone who knows more about DBus and how it is used, please do correct any bad assumptions or errors of fact I have made in this post!

TOMORROW: those Gnome bits I have got working, and how; those Gnome bits I have not got working, and wild speculation as to why...

[hypeitnow]

Hi,

First of all great work. Right now I am trying to solve a problem regarding github token and I stuck at one moment.

Is it somehow possible to make the wlsg DBUS process accessible to the current user's bash session?

Thanks in advance

[cerebrate]

On one hand, I haven't found a way to do this, either by bridging the two system dbus-daemons, or by finding a way to extend the system distro dbus-daemon into the WSL distro namespace. It seems somewhat tricky in the first instance since WSL distros don't run dbus-daemon without some hackery, and I suspect most people with such a configuration are getting there by running systemd, which is itself entangled with dbus in various ways and may not cooperate with a "foreign" system instance.

On the other hand, I don't know dbus all that well, and looking at the system distro dbus instance as I did above shows that there's nothing actually connected to it, and so starting your own dbus-daemon system instance ought work every bit as well. (The system distro does not include a dbus-daemon session instance.) Given that, I confess I haven't been looking for one.

tl;dr If there is one, I don't know what it is, but as far as I can tell there's nothing to be gained by doing so anyway.