Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instance upgraded from focal to noble is missing systemd-resolved #7464

Open
legoktm opened this issue Mar 6, 2025 · 6 comments · May be fixed by #7466
Open

Instance upgraded from focal to noble is missing systemd-resolved #7464

legoktm opened this issue Mar 6, 2025 · 6 comments · May be fixed by #7466

Comments

@legoktm
Copy link
Member

legoktm commented Mar 6, 2025

Description

Doing an in-place upgrade to noble results in the system missing systemd-resolved. A fresh noble install will still have it.

Steps to Reproduce

  1. Set up focal 2.12.0-rc.2
  2. Do a semiautomated / fully automated upgrade
  3. On a server, run systemctl status systemd-resolved - it'll error
  4. Try running ./securedrop-admin install - it will error on the step about stopping systemd-resolved.

Expected Behavior

Either:

  1. systemd-resolved is present on upgraded systems
  2. Or, systemd-resolved is removed from all noble systems

And, ./securedrop-admin install should work post-upgrade.

Next steps

  1. diff the output of dpkg -l from both a fresh noble install and an upgrade and see if anything else is missing (ideally check both app and mon)
  2. figure out what to do with systemd-resolved. Not sure why it's getting removed during upgrade, a missing dependency maybe? We could fix that, or just say that we don't actually need it since we stop it and uninstall it on fresh installs too. The former is probably safer.
@zenmonkeykstop
Copy link
Contributor

dpkg -l runs from an upgraded and freshly installed noble instance:

upgrade_app_dpkg.txt

upgrade_mon_dpkg.txt

fresh_app_dpkg.txt

fresh_mon_dpkg.txt

@legoktm
Copy link
Member Author

legoktm commented Mar 6, 2025

Thanks, I extracted the raw package lists (grep "ii" fresh_mon_dpkg.txt | awk '{print $2}') and then diffed them:

Looking up all the individual packages now.

@legoktm
Copy link
Member Author

legoktm commented Mar 6, 2025

  • appstream: don't need it
  • e2fsprogs-l10n: don't need it
  • firmware-sof-signed: firmware for audio devices - don't need it
  • ibverbs-providers: userspace drivers for specialized ethernet hardware. My guess is that NUCs don't need this, but will examine a bit more.
  • jq: don't need it? (only used in dev AFAICT)
  • libgpg-error-l10n: don't need it, just translations

[skipping all the other lib* packages because they're dependencies]

  • lxd-installer: don't need it
  • needrestart: don't need it
  • nftables: I believe this is just the CLI frontend? don't need it but might be nice to have.
  • numactl: don't need it
  • python3-boto3, python3-botocore, python3-s3transfer: don't need it
  • python3-dateutil: don't need it
  • python3-jmespath: don't need it
  • python3-markdown-it, python3-mdurl: don't need it
  • python3-pygments: don't need it
  • python3-rich: don't need it
  • sysstat: don't need it
  • systemd-hwe-hwdb: "udev rules for hardware enablement", I don't know what this is and it's an Ubuntu-specific thing.
  • systemd-resolved: !!!
  • trace-cmd: don't need it

On mon specifically, we have:

-media-types
+mime-support

media-types replaced mime-support, but we missed the transitional package, so we'll have to do it manually. Not sure what dependency is pulled it in though and why app wouldn't have it.

@legoktm
Copy link
Member Author

legoktm commented Mar 6, 2025

In focal, the systemd-resolved component was included in the main systemd package:

$ dpkg -c systemd_245.4-4ubuntu3.24_amd64.deb | grep resolved
-rwxr-xr-x root/root      3786 2024-06-17 20:29 ./etc/dhcp/dhclient-enter-hooks.d/resolved
-rw-r--r-- root/root       642 2024-06-17 20:29 ./etc/systemd/resolved.conf
-rw-r--r-- root/root      1731 2024-06-17 20:29 ./lib/systemd/system/systemd-resolved.service
-rwxr-xr-x root/root    415968 2024-06-17 20:29 ./lib/systemd/systemd-resolved
-rw-r--r-- root/root      4227 2024-06-17 20:29 ./usr/share/man/man5/resolved.conf.5.gz
-rw-r--r-- root/root      4512 2024-06-17 20:29 ./usr/share/man/man8/systemd-resolved.service.8.gz
lrwxrwxrwx root/root         0 2024-06-17 20:29 ./usr/share/man/man5/resolved.conf.d.5.gz -> resolved.conf.5.gz
lrwxrwxrwx root/root         0 2024-06-17 20:29 ./usr/share/man/man8/systemd-resolved.8.gz -> systemd-resolved.service.8.gz

This meant there was no way to uninstall it, so we only stopped it.

In noble, it was split out into a separate package. But nothing explicitly depends on it, so we don't install it during upgrade, and lose it.

Knowing that, my preference would be to have the securedrop-config package gain a dependency on systemd-resolved, but only for the noble version. We already have similar logic in app-code for this. This is a small, self-contained change that's easy to statically verify by looking at the package metadata. And then running through the actual upgrade. Once everyone is on noble, we can remove the systemd-resolved package across the board.

I can put a PR in for this after figuring out the systemd-hwe-hwdb thing and nftables.


On the QA side, I think we should add a step like, "After an in-place upgrade, all the ./securedrop-admin commands still work, e.g. install, backup, etc."

@legoktm
Copy link
Member Author

legoktm commented Mar 6, 2025

On mon specifically, we have:

-media-types
+mime-support

On my upgraded mon server, I see mime-support being removed, so it's weird that yours still has it.

@legoktm
Copy link
Member Author

legoktm commented Mar 6, 2025

Currently systemd-hwe-hwdb contains exactly one udev rule:

# Dell Pro Rugged microphone mute
evdev:name:Dell WMI hotkeys:dmi:bvn*:bvr*:bd*:svnDell*:pnDellProRugged*:*
 KEYBOARD_KEY_100150=f20

I think it's easiest just to install it the same way we're going to do with systemd-resolved.

@legoktm legoktm self-assigned this Mar 6, 2025
legoktm added a commit that referenced this issue Mar 6, 2025
These two packages are installed on fresh systems, but not on upgrades
because they were split out of the systemd package. Set the dependency
ourselves to make sure it's always pulled in.

In the future once all SecureDrops are on noble, we can uninstall
systemd-resolved entirely instead of merely stopping it.

Fixes #7464.
legoktm added a commit that referenced this issue Mar 6, 2025
These two packages are installed on fresh systems, but not on upgrades
because they were split out of the systemd package. Set the dependency
ourselves to make sure it's always pulled in.

In the future once all SecureDrops are on noble, we can uninstall
systemd-resolved entirely instead of merely stopping it.

Fixes #7464.
legoktm added a commit that referenced this issue Mar 7, 2025
These two packages are installed on fresh systems, but not on upgrades
because they were split out of the systemd package. Set the dependency
ourselves to make sure it's always pulled in.

In the future once all SecureDrops are on noble, we can uninstall
systemd-resolved entirely instead of merely stopping it.

Fixes #7464.
legoktm added a commit that referenced this issue Mar 7, 2025
We don't use systemd-resolved and during the focal -> noble migration,
it gets dropped since it was split out to a separate package.

Now that we can remove it entirely, let's have noble installs absent the
package instead of merely stopping the systemd unit.

Fixes #7464.
legoktm added a commit that referenced this issue Mar 7, 2025
We don't use systemd-resolved and during the focal -> noble migration,
it gets dropped since it was split out to a separate package.

Now that we can remove it entirely, let's have noble installs absent the
package instead of merely stopping the systemd unit.

Fixes #7464.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants