systemd status fails output on Ubuntu
#8879
Comments
|
Yeah, this one's been happening since systemd 250 shipped, under all the systemd hacks and now under the official support. |
|
I've done some investigating: I see 3 possible places the bug might be:
|
|
I think it is 3; that's the same root cause @nullpo-head figured out for the problem when it was reported under distrod (nullpo-head/wsl-distrod#31 (comment)), with the same fix (put systemd in a new cgroup). Of course, with systemd now running as pid 1 without a hack, the fix'll need to be in WSL proper. |
|
Still an issue on 0.68.2, FWIW. |
|
Still an issue also on the latest 1.0.0 $ wsl --version
Versione WSL: 1.0.0.0
Versione kernel: 5.15.74.2
Versione WSLg: 1.0.47
Versione MSRDC: 1.2.3575
Versione Direct3D: 1.606.4
Versione DXCore: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
versione Windows: 10.0.19045.2311 |
|
I've applied the following workaround to my systemd on WSL, and it makes the output look like (I think) it's supposed to look: It's most likely not a proper fix or a good idea to do, but it gets stuff working for me in the meantime. |
Thanks. I also rebuilt the This is on |
|
This is also happening on Fedora 36: systemd version: WSL version: |
|
This is also happening on Kali-linux |
|
I guess this issue will get more attention over time with more distos update systemd to >= 250. At the moment it seems that the only supported distro with such an systemd version is Kali (but I did not verify this). |
|
The patch provided by @BtbN does work for me on Fedora 36 using the technique described by @chutzimir. |
|
Using different systemd versions openSUSE Tumbleweed with systemd 252.5, not working: Ubuntu 22.04 with systemd 249.11, working: Both distros run on: |
Works and was dead easy to get into NixOS: nixpkgs.overlays = [
(final: prev: with final; {
systemd-wsl = final.systemd.overrideAttrs ({ patches, ... }: {
patches = patches ++ [
./systemd-systemctl-status-wsl.patch
];
});
})
];
systemd.package = pkgs.systemd-wsl; |
|
Still an issue to date. Also tested with pre-release 1.2.4. Both using cgroups v2. Of the Ubuntu apps, that situation still only affects the Ubuntu-Preview for now, but eventually other releases will be at systemd version 2.50 or later, so that will be a problem. A suitable fix seems to be to create and assign a cgroup and a cgroup namespace to every distro instance. Handling the cgroup2 mount point properly before starting systemd, it should then see that as its root cgroup. There will be no PID 0's inside the For the sake of demonstration, if instead of enabling systemd through the proper settings in
#!/bin/sh
set -eu
# Ensure that we don't clean up /tmp setup by the Microsoft init process.
mkdir -p /run/tmpfiles.d
ln -s /dev/null /run/tmpfiles.d/tmp.conf
umount /proc/sys/fs/binfmt_misc
# Create the distro cgroup
mkdir -p /sys/fs/cgroup/distros/ubuntu-preview
# Start systemd in its own namespace. Notice the `-C` for cgroup
env -i /usr/bin/unshare -Cm --fork --mount-proc --pid --propagation unchanged -- sh -c '
echo $PPID > /sys/fs/cgroup/distros/ubuntu-preview/cgroup.procs && exec /lib/systemd/systemd --unit=multi-user.target
' &DISCLAIMER: By no means the above is a solution. It has many drawbacks, such as not showing the systemd root cgroup as Configuring the boot command parameter to run that script and starting a shell under systemd namespaces allows us to see the following
[boot]
command=<absolute path of the script> |
|
This issue is reproducible on WSL-based Arch Linux with $ sudo SYSTEMD_LOG_LEVEL=7 SYSTEMD_PAGER= systemctl status
Bus n/a: changing state UNSET → OPENING
sd-bus: starting bus by connecting to /run/systemd/private...
Bus n/a: changing state OPENING → AUTHENTICATING
Bus n/a: changing state AUTHENTICATING → RUNNING
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 path=/org/freedesktop/systemd1 interface=org.freedesktop.DBus.Properties member=GetAll cookie=1 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=method_return sender=org.freedesktop.systemd1 destination=n/a path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=a{sv} error-name=n/a error-message=n/a
● MSI-B660M
State: running
Units: 252 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 0 units
Since: Thu 2023-11-09 11:18:50 PST; 25min ago
systemd: 254.5-1-arch
Tainted: cgroupsv1
CGroup: /
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=GetUnitProcesses cookie=2 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=error sender=org.freedesktop.systemd1 destination=n/a path=n/a interface=n/a member=n/a cookie=2 reply_cookie=2 signature=s error-name=org.freedesktop.DBus.Error.IOError error-message=Input/output error
Failed to dump process list for 'MSI-B660M', ignoring: Input/output error
Bus n/a: changing state RUNNING → CLOSED |
|
The same with debian |
|
Gentoo. |
|
Still an issue with Oracle Linux 9.2 installed and updated from the Microsoft Store: |
|
Updated patch for v255: diff --git a/src/core/dbus-unit.c b/src/core/dbus-unit.c
index 1a037b7..f0550e8 100644
--- a/src/core/dbus-unit.c
+++ b/src/core/dbus-unit.c
@@ -1300,6 +1300,8 @@ static int append_cgroup(sd_bus_message *reply, const char *p, Set *pids) {
* readable in the subtree proper. */
r = cg_read_pidref(f, &pidref);
+ if (r == -EIO)
+ continue;
if (IN_SET(r, 0, -EOPNOTSUPP))
break;
if (r < 0) |
|
Also seeing this. > wsl --version
WSL-Version: 2.1.5.0
Kernelversion: 5.15.146.1-2
WSLg-Version: 1.0.60
MSRDC-Version: 1.2.5105
Direct3D-Version: 1.611.1-81528511
DXCore-Version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows-Version: 10.0.22631.3447> systemctl --version
systemd 252 (252.22-1~deb12u1)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified❯ PAGER= SYSTEMD_LOG_LEVEL=debug systemctl status
Bus n/a: changing state UNSET → OPENING
sd-bus: starting bus by connecting to /run/systemd/private...
Bus n/a: changing state OPENING → AUTHENTICATING
Bus n/a: changing state AUTHENTICATING → RUNNING
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 path=/org/freedesktop/systemd1 interface=org.freedesktop.DBus.Properties member=GetAll cookie=1 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=method_return sender=org.freedesktop.systemd1 destination=n/a path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=a{sv} error-name=n/a error-message=n/a
● metro
State: degraded
Units: 282 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 2 units
Since: Thu 2024-04-25 23:12:44 CEST; 16min ago
systemd: 252.22-1~deb12u1
CGroup: /
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=GetUnitProcesses cookie=2 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=error sender=org.freedesktop.systemd1 destination=n/a path=n/a interface=n/a member=n/a cookie=2 reply_cookie=2 signature=s error-name=org.freedesktop.DBus.Error.IOError error-message=Input/output error
Failed to dump process list for 'metro', ignoring: Input/output error
Bus n/a: changing state RUNNING → CLOSED |
@MorphBonehunter, yep. Ubuntu 24 is around the corner and it uses systemd 255 |
|
Yes, do not update to Ubuntu 24.04, I made that mistake yesterday. |
|
This issue is reproducible on WSL-based Ubuntu Linux with systemd 253.5-1ubuntu6.1. |
In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from other instances under the same WSL Kernel, which at least always hosts the system instance with the X/Wayland/PA/Pipe server. Without this patch, whenever cg_read_pid encounters such a zero, it throws an error. This makes systemd near unusable inside of WSL. Change cg_read_pid() to return 0, and adjust all callers to handle that appropriately. In general, we cannot do anything with such processes, so most operations have to be refused. The only thing we can do with them is count them, in particular, we can answer the question whether the cgroup is empty in the negative. On normal systems, where the list does not contain any zeros to begin with, this has no averse effects. See also: microsoft/WSL#8879. Co-authored-by: Timo Rothenpieler <[email protected]>
In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from other instances under the same WSL Kernel, which at least always hosts the system instance with the X/Wayland/PA/Pipe server. Without this patch, whenever cg_read_pid encounters such a zero, it throws an error. This makes systemd near unusable inside of WSL. Change cg_read_pid() to return 0, and adjust all callers to handle that appropriately. In general, we cannot do anything with such processes, so most operations have to be refused. The only thing we can do with them is count them, in particular, we can answer the question whether the cgroup is empty in the negative. On normal systems, where the list does not contain any zeros to begin with, this has no averse effects. This replaces systemd#32534. See also: microsoft/WSL#8879. Co-authored-by: Timo Rothenpieler <[email protected]>
In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from other instances under the same WSL Kernel, which at least always hosts the system instance with the X/Wayland/PA/Pipe server. Without this patch, whenever cg_read_pid encounters such a zero, it throws an error. This makes systemd near unusable inside of WSL. Change cg_read_pid() to return 0, and adjust all callers to handle that appropriately. In general, we cannot do anything with such processes, so most operations have to be refused. The only thing we can do with them is count them, in particular, we can answer the question whether the cgroup is empty in the negative. On normal systems, where the list does not contain any zeros to begin with, this has no averse effects. This replaces systemd#32534. See also: microsoft/WSL#8879. Co-authored-by: Timo Rothenpieler <[email protected]>
In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from other instances under the same WSL Kernel, which at least always hosts the system instance with the X/Wayland/PA/Pipe server. Without this patch, whenever cg_read_pid encounters such a zero, it throws an error. This makes systemd near unusable inside of WSL. Change cg_read_pid() to return 0, and adjust all callers to handle that appropriately. In general, we cannot do anything with such processes, so most operations have to be refused. The only thing we can do with them is count them, in particular, we can answer the question whether the cgroup is empty in the negative. On normal systems, where the list does not contain any zeros to begin with, this has no averse effects. This replaces systemd#32534. See also: microsoft/WSL#8879. Co-authored-by: Timo Rothenpieler <[email protected]>
In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from other instances under the same WSL Kernel, which at least always hosts the system instance with the X/Wayland/PA/Pipe server. Without this patch, whenever cg_read_pid encounters such a zero, it throws an error. This makes systemd near unusable inside of WSL. Change cg_read_pid() to return 0, and adjust all callers to handle that appropriately. In general, we cannot do anything with such processes, so most operations have to be refused. The only thing we can do with them is count them, in particular, we can answer the question whether the cgroup is empty in the negative. On normal systems, where the list does not contain any zeros to begin with, this has no averse effects. This replaces systemd#32534. See also: microsoft/WSL#8879. Co-authored-by: Timo Rothenpieler <[email protected]>
|
This is fixed in the latest systemd stable version of 255, so someone just has to get Ubuntu to update. |
I am still seeing the same error message on 255 (fedora 40): Is there something else I'm doing wrong? |
|
@MorphBonehunter got it, thank you! I misunderstood @BtbN and thought every 255 version fixed the problem, didnt notice that it was fixed in a minor release. Thanks! EDIT: can confirm, updating to 255.7 fixed the issue for me! |
|
same issue here |
|
In Ubuntu, we have published a patched version of systemd 255.4-1ubuntu8.2 to the noble-proposed repository. |
|
Seems to work fine also here on Arch. CMD output: ❯ wsl --version
Versione WSL: 2.2.4.0
Versione kernel: 5.15.153.1-2
Versione WSLg: 1.0.61
Versione MSRDC: 1.2.5326
Versione Direct3D: 1.611.1-81528511
Versione DXCore: 10.0.26091.1-240325-1447.ge-release
Versione di Windows: 10.0.22631.3880zsh output: ❯ systemctl status
● ASUS-ROG-LAPTOP
State: running
Units: 302 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 0 units
Since: Wed 2024-07-10 20:23:28 CEST; 2min 5s ago
systemd: 256.2-1-arch
Tainted: cgroupsv1
CGroup: /
#[...] Thanks! |
I'm on systemd 255 (255.4-1ubuntu8.2) and it looks like it is still broken, Running
Still getting a lot of If I then do |
|
I've been informed on Twitter that the fix should be in 255.7 |
|
@Stanzilla On Ubuntu 24.04: $ sudo systemctl --version 1.5m Mon 21:33
systemd 256 (256-1ubuntu1)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT +LIBARCHIVEStill doesn't work. |
Yikes. I hope we get some better news soon. Should we open a bug at Ubuntu? @craigloewen-msft @benhillis? |
Could check or re-open this one: That one says it was fixed in |
|
It now works for me |
|
You need to do a daemon-reexec or just plain wsl restart after installing the update. |
I did that several times actually |
is that a fresh install or something before 24.04 that you upgraded? |
|
Installed the distribution 2024-04-29, grabbed the image for 24.04 directly from Ubuntu. Back then it was not yet available in the Microsoft Store |
and others
Version
10.0.22621.521
WSL Version
Kernel Version
5.15.62.1
Distro Version
Ubuntu 22.10
Other Software
No response
Repro Steps
install Ubuntu (Preview), store id 9P7BDVKVNXZ6. This version has systemd already enabled.
Expected Behavior
systemctl statuscommand to workActual Behavior
It doesn't work on ArchLinux also:
Both WSL instances run
systemd-251.4, so this could be asystemdissue. It works okay on Ubuntu 22.04 andsystemd-249.11.Diagnostic Logs
No response
The text was updated successfully, but these errors were encountered: