Debian Bookworm testing and upgrade script feedback

[MichaIng]

This issue aims to collect feedback, testing results and info around the upcoming Debian Bookworm release.

Please read our article to get an overview: T.b.d.

Additional info can be found on our wiki: https://github.com/MichaIng/DietPi/wiki/Debian-Bookworm-testing

To run the Bookworm migration script on Bullseye systems:

sudo bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/DietPi/master/.meta/dietpi-bookworm-upgrade')

[manilx]

Used the script to update 6 VM's. On 4 it worked without issue on 2 it errored out.

Strange thing is they all had a base template......

[manilx]

Setting up grub-pc (2.06-8) ...
Replacing config file /etc/default/grub with new version
grub-pc: Running grub-install ...
/dev/sda does not exist, so cannot grub-install to it!
You must correct your GRUB install devices before proceeding:

  DEBIAN_FRONTEND=dialog dpkg --configure grub-pc
  dpkg --configure -a
dpkg: error processing package grub-pc (--configure):
 installed grub-pc package post-installation script subprocess returned error exit status 1

[Joulinar]

can you share following from the 2 VM's having an issue and another example from a system working

lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid

[manilx]

Working:

NAME    FSTYPE LABEL  SIZE RO TYPE MOUNTPOINT PARTUUID                             UUID
xvda                    8G  0 disk
|-xvda1 ext4            7G  0 part /          42e48e80-01                          7e5234cc-eb02-4544-b50b-9b14cda36655
|-xvda2                 1K  0 part            42e48e80-02
└─xvda5 swap          975M  0 part            42e48e80-05                          b77c54c2-e515-4c40-b2a5-6297b1a2a944

Not working:

NAME    FSTYPE LABEL  SIZE RO TYPE MOUNTPOINT PARTUUID                             UUID
xvda                    8G  0 disk
├─xvda1 ext4            7G  0 part /          42e48e80-01                          7e5234cc-eb02-4544-b50b-9b14cda36655
├─xvda2                 1K  0 part            42e48e80-02
└─xvda5 swap          975M  0 part            42e48e80-05                          b77c54c2-e515-4c40-b2a5-6297b1a2a944

[manilx]

Sorry. This is the non working.

NAME FSTYPE LABEL SIZE RO TYPE MOUNTPOINT PARTUUID UUID
xvda 8G 0 disk
|-xvda1 ext4 7G 0 part / 42e48e80-01 7e5234cc-eb02-4544-b50b-9b14cda36655
|-xvda2 1K 0 part 42e48e80-02
`-xvda5 swap 975M 0 part 42e48e80-05 b77c54c2-e515-4c40-b2a5-6297b1a2a944

Working:
NAME FSTYPE LABEL SIZE RO TYPE MOUNTPOINT PARTUUID UUID
xvda 8G 0 disk
├─xvda1 ext4 7G 0 part / 42e48e80-01 7e5234cc-eb02-4544-b50b-9b14cda36655
├─xvda2 1K 0 part 42e48e80-02
└─xvda5 swap 975M 0 part 42e48e80-05 b77c54c2-e515-4c40-b2a5-6297b1a2a944
There's a difference in xvda5.

[manilx]

Actually that swap is not needed... Guess it's a leftover off debian 11 install (installed it from debian 11 iso) and then converted via dietpi script.

[MichaIng]

The GRUB package needs to know where it can flash GRUB to. Since we cannot know the device name you flash the image to, we use ship them with /dev/sda as most common root device name. On first GRUB update, you however needed to change this manually, if it was wrong. Looks like you did this on 4 VMs already, but not yet on 2 of them. Run manually once:

dpkg --configure grub-pc

and select the correct device name, which seems to be /dev/xvda in your case.

Since DietPi v7.5, it is adjusted automatically on first boot: d430094
So probably your two affected VMs are the only older ones. Or if the others are older as well, probably the root device name changed after first boot, e.g. if you changed some settings on the hypervisor or moved the drive to another VM.

[manilx]

Tried to run, got:

# dpkg --configure grub-pc dpkg: error processing package grub-pc (--configure): package grub-pc is already installed and configured Errors were encountered while processing: grub-pc

[MichaIng]

Try

dpkg --configure -a

or

apt -f install

or

apt update
apt install grub-pc

instead. Not sure which state your packages are exactly on each VM.

[manilx]

Doesn't help.

┌──(root💀Wireguard)-[~]
└─# apt -f install
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
┌──(root💀Wireguard)-[~]
└─# apt update
apt install grub-pc
Hit:1 https://deb.debian.org/debian bullseye InRelease
Get:2 https://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Hit:3 https://deb.debian.org/debian-security bullseye-security InRelease
Get:4 https://deb.debian.org/debian bullseye-backports InRelease [49.0 kB]
Fetched 93.0 kB in 1s (97.8 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
grub-pc is already the newest version (2.06-3~deb11u5).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
┌──(root💀Wireguard)-[~]
└─# dpkg --configure -a
┌──(root💀Wireguard)-[~]
└─# dpkg --configure grub-pc
dpkg: error processing package grub-pc (--configure):
 package grub-pc is already installed and configured
Errors were encountered while processing:
 grub-pc

[MichaIng]

This is on a system where the Bookworm upgrade script just failed? Seems all fine. You can force the configuration with

dpkg-reconfigure grub-pc

[Joulinar]

Btw apt package source seems to be bullseye still.

[manilx]

This a bullseye system before the upgrade script (rolled back) because the update script was failing (this is why I reported here as asked on the announcement).

[Joulinar]

But you have the issue with Grub already on the Bullseye system?

[manilx]

No issues whatsoever! Runs fine.

[manilx]

Tried the upgrade script again. Errored out.
Entered "dpkg-reconfigure grub-pc" and got

/usr/sbin/dpkg-reconfigure: grub-pc is broken or not fully installed

[manilx]

Now I entered "dpkg --configure -a" and I could choose the device.

Seems OK now.

[Joulinar]

But as shown above, dpkg --configure grub-pc was failing on Bullseye?

[manilx]

Yes.

[Joulinar]

This would mean the issue was already there on Bullseye and just popped up due to the Bookworm upgrade. Right?

[manilx]

Bullseye was running fine and I had no issue booting..... So I can't tell if there was an issue.
Tried the upgrade script and it failed.

[MichaIng]

Makes all sense then. You could have done dpkg-reconfigure grub-pc already before running the script, to fix the debconf entry in the first place.

[manilx]

OK.... Doesn't make to me but then. As I was not experiencing any issue booting.
Anyway fixed now. Can't tell what /when started it.

[Phil1988]

Ok maybe there is a problem when upgrading.
Every reboot results in a "stuck while booting" problem.
I can see the curser blinking, but its not booting - not even the BIOS messages being visible.

This is always when doing a reboot from terminal/console.
Its OK, when rebooting from the VM control panel (which simulates an hardware reboot)

I then always see this while booting just before the login screen pops up:

But this might not be related

[MichaIng]

Which virtualizer is it? Probably check its settings, use latest hardware/controller types etc. Usually with GRUB the very first output is "Loading GRUB", followed by "Loading initrd" or similar. If there is really nothing visible at all, then this is either on a hardware/emulation level where software should not have any effect on, or it is not a boot issue, but a shutdown issue. Probably some service is hanging. By default, it should be killed after 90 seconds.

You could enable persistent system logs and see whether something is caught:

dietpi-software uninstall 103
mkdir /var/log/journal
reboot

Persistent logs should be enabled after this reboot. So do another reboot then, and check via journalctl.

If you did use Unbound, e.g. behind Pi-hole, then you would recognise it being down, since no client would be able to resolve any hostname. So I guess it is currently not in use anyway? So you could just uninstall it, or if needed, check where the error is coming from:

journalctl -u unbound

[buttonpushertv]

The script is requesting a 'dist' file from a server that doesn't have any 'bookworm' dist files. (Manually visiting pkg/ltec.ch/public/dists shows there is no bookworm folder).

Is there another repo that hosts them? Or should the script be updated?

APT update
 - Command: apt-get -y -eany update
 - Exit code: 100
 - DietPi version: v8.25.1 (MichaIng/master) | HW_MODEL: 3 | HW_ARCH: 2 | DISTRO: 6
 - Image creator: DietPi Core Team
 - Pre-image: Raspbian Lite
 - Error log:
Hit:2 http://raspbian.raspberrypi.org/raspbian bookworm InRelease
Hit:1 https://linux-packages.resilio.com/resilio-sync/deb resilio-sync InRelease
Hit:3 https://archive.raspberrypi.org/debian bookworm InRelease
Ign:4 https://download.webmin.com/download/newkey/repository stable InRelease
Hit:5 https://download.webmin.com/download/newkey/repository stable Release
Ign:6 https://pkg.ltec.ch/public bookworm InRelease
Err:7 https://pkg.ltec.ch/public bookworm Release
  404  Not Found [IP: 95.143.48.181 443]
Reading package lists...
E: The repository 'https://pkg.ltec.ch/public bookworm Release' does not have a Release file.

[Joulinar]

You would need to manually switch back this particular repository to Bullseye.

[buttonpushertv]

Found this forum post (that I must've overlooked) when searching for more info:

https://dietpi.com/forum/t/dietpi-update-error-with-release-file-pkg-ltec-ch-bookworm/17519/2

The solution here:
https://dietpi.com/forum/t/dietpi-update-error-with-release-file-pkg-ltec-ch-bookworm/17519/6

wsdd is included in bookworm package now. You can remove the source for in apt:
sudo rm /etc/apt/sources.list.d/dietpi-wsdd.list

[wildding-github]

It might be useful to update the section on wsdd in the DietPi Samba guide (https://dietpi.com/docs/software/file_servers/#__tabbed_2_5) accordingly to prevent problems during the update.

[Joulinar]

I guess we need to differentiate between Bullseye and Bookworm. On Bullseye it will be dedicated pkg.ltec.ch repository still, while on Bookworm it has become part of global Debian repository.

[MichaIng]

But thanks for the hint! Now I know why the lists have dietpi- suffix. I was forgetting about this docs entry.

Btw, I recognised just now that mkdocs-material respectively pymdownx.tabbed added support for opening a particular tab via URL fragment. That is awesome and allows us better cross referencing specific content. @StephanStS also to ping you here. There are options to e.g. include the header of the block into the fragment of the tab, to make it more meaningful: https://squidfunk.github.io/mkdocs-material/setup/extensions/python-markdown-extensions/?h=tabbed#+pymdownx.tabbed.combine_header_slug

EDIT: PR to update docs: MichaIng/DietPi-Docs#961

[wildding-github]

But thanks for the hint! Now I know why the lists have dietpi- suffix. I was forgetting about this docs entry.

Without the ingenious DietPi OS, I would not have been able to give this hint.

[ElDavoo]

Just FYI: I updated my Raspberry Pi 3B and for some reason it rebooted during the apt upgrade phase.
So I just ran dpkg --configure -a and then re-ran the script.
All worked fine, but the raspberry didn't come up.
Luckily, unplugging and replugging the power did the trick.
Looks like that for some reason typing reboot makes the raspberry stuck. So not a problem with this script :)

[MichaIng]

Probably some service was hanging on shutdown. A reboot during APT upgrades should not happen, so probably a power/voltage issue, or theoretically emergency shutdown due to critical temperature. Keep an eye on:

dmesg -l 0,1,2,3

whether you see some voltage related kernel errors.

[Frankynov]

Hello everyone,
I have performed the update yesterday evening to Bookworm. Globally it went fine, however I have noticed two issues so far :

• The nginx configuration seems to have reset itself back to default. For example, the file /etc/nginx/sites-available/default didn't have any of my configuration. A simple restore from the backup location of that file solved the issue
• The Home assistant installation doesn't seem to start well. I have a lot of errors in the web interface, such as :
  `Logger: homeassistant
  Source: setup.py:464
  First occurred: 08:38:04 (1 occurrences)
  Last logged: 08:38:04

Error doing job: Task exception was never retrieved
`
Still unknown what to do here, will attempt a clean reinstall of home assistant.
<edit : with this workaround #6931 I could boot home assistant properly, will need however to repair all my bluetooth devices I think

• At first the bash wouldn't allow to use autocompletion when a space was used in a command, getting error : bash: syntax error near unexpected token ('`
  After a reboot it looks like it's solved

[Depechie]

I'm trying the script to perform the upgrade from Bullseye to Bookworm, but it 'hangs' on the APT dist-upgrade part.
Canceling the upgrade and performing the suggested apt autoremove has no effect.

Weird part is that the list of hits have a mix of bullseye and bookworm?

[Joulinar]

How long did you wait? At this point some packages are downloaded. Sometimes this could take a while depending on apt package server speed.

As well the mixture of Bullseye and Bookworm might be ok. Some apps don't offer a Bookworm repository. It depends on the individual apps.

[Depechie]

@Joulinar I did not time it. I will restart the procedure and wait at least 1 hour and report back!

[Depechie]

@Joulinar ok sorry for the inconvenience, I indeed just needed to wait longer... ( thx for the feedback )

[MichaIng]

What takes so long is collecting/calculating all package/upgrade information without APT cache and compressed package lists for such large upgrades (large number of packages to upgrade), indeed. APT seems to do this pretty ineffective, at least it definitely does not uncompress all lists first, then all parsing and calculation, but seems to do decompression/parsing for each and every package, or in chunks, over and over again. In dietpi-config advanced options > APT, the binary APT cache can be enabled, as well as list compression disabled. However, when not stored in RAM (which of course raises RAM usage), it implies significant disk writes on every apt update, hence probably not advised when using an older/cheaper SD card for the root filesystem.

The apt autoremove btw is done on first (root) login after reboot, which is more failsafe than doing this immediately, as in some cases modules for the currently loaded kernel would be removed. I would remove this output from APT, but I am not aware of a way to hide this without omitting other processing output as well, which would mean longer phases of no output, causing more often the impression that APT hangs.

[MichaIng]

I'll close this now. For any issue or question regarding the script or Bookworm in general, please open a new issue.