Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image | Raspberry Pi 5: Testing and firmware migration script #6676

Open
TDuffinNTU opened this issue Oct 9, 2023 · 362 comments · Fixed by #6795 or #6892
Open

Image | Raspberry Pi 5: Testing and firmware migration script #6676

TDuffinNTU opened this issue Oct 9, 2023 · 362 comments · Fixed by #6795 or #6892

Comments

@TDuffinNTU
Copy link
Contributor

TDuffinNTU commented Oct 9, 2023

ADMIN EDIT

First Raspberry Pi 5 testing images are available now on our download page: https://dietpi.com/#download
You can also find images with the new kernel/firmware package set for other RPi models here: https://dietpi.com/downloads/images/testing/
The ones with the new firmware have "RPi1", "RPi2" and "RPi234" (64-bit) in their names. To migrate an existing system, use the migration script:

bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')

WARNING: This is currently a one-way ticket. dietpi-backup cannot be used to restore the old system, since the boot partition mount point has changed. If you want to be able to revert, create an image of the whole SD card/drive. dietpi-imager can be used from another DietPi (or Debian/Ubuntu) system, to minimise partition and filesystem automatically, to have a small compressed backup image.


Is the SBC officially supported by the Debian installer?

  • Yes(?) Looks like the board is inter-compatible with the same distro images as the 4B. That's pure speculation, mind you, but I did see a video of a reviewer transferring a Raspbian installation from a Pi4B to a Pi5 with no configuration necessary.

Notes

  • Not sure if this needs any specific developer legwork as the previous Pis seem to be inter-compatible with OS images for the most part (though I'm sure testing and drivers will be necessary!). Thought I'd make this issue now since it might be good to mark the Pi5 as provisionally supported on the website if it at least works out of the box.
@MichaIng MichaIng changed the title Raspberry Pi 5 Image | Raspberry Pi 5 Oct 10, 2023
@MichaIng
Copy link
Owner

MichaIng commented Oct 19, 2023

Okay this requires more work, sadly. With Bookworm and hence for RPi 5, the packages as well as the filesystem layout have changed dramatically:

  • For RPi 4 and earlier, the package is called linux-image-rpi-v8, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-v8.
  • For RPi 5, the package is called linux-image-rpi-2712, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-2712.
  • These packages are more streamlined with mainline Linux packages, installing kernel image, config, map, device trees and overlays like common Debian Linux packages:
    /boot/System.map-6.1.0-rpi4-rpi-2712
    /boot/config-6.1.0-rpi4-rpi-2712
    /boot/vmlinuz-6.1.0-rpi4-rpi-2712
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/broadcom/bcm2712-rpi-5-b.dtb
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/overlays/README
    
  • For the bootloader, there is now a raspi-firmware package, which installs the files as well more streamlined into /usr/lib/raspi-firmware/.
  • libraspberrypi0, libraspberrypi-bin and raspberrypi-sys-mods remain the same.
  • The RPi ROM bootloader stage still requires bootloader, config and kernel on a partition 1 with FAT filesystem, which was mounted to /boot. This cause the issue that, since dpkg does not support FAT, this rpikernelhack back and forth copying of all files in /boot was required, to bypass the need to dpkg to replace a file (which would fail). This procedure has now been replaced with another, IMO not less ugly workaround: The FAT filesystem of partition 1 is now mounted to /boot/firmware. The raspi-firmware package strictly requires this directory to be a mount point (now allowing a dir within a mount point) and then copies all kernel, dtb and bootloader files from /boot and /usr/lib/linux-image- over to /boot/firmware, applying the known naming schemes. So a copy back and forth of all those files has been replaced with a single copy, but files remaining permanently on both partitions. And of course, for the 64-bit kernel, it is now actually two kernel images and all related modules, hence disk space requirements have increased.
  • Furthermore, the raspi-firmware package does now strictly require an initramfs. I have to check whether this means that the kernel has essential modules no builtin anymore, or whether it is actually not required, but just installed and handled for in case. As far as I can see, there is a config flag to disable this copy to /boot/firmware stuff, which then would need to be done manually. So we could write our own script to do this, skipping the initramfs. But at least at the beginning, I am not keen to maintain such, as it will likely change a lot within the next months. Probably a tiny-initramfs is possible, like the moreless dummy one we do now use for Sparky SBC, one which does not include any kernel module, hence servers nearly no purpose (aside of supporting UUIDs on top of PARTUUIDs) aside of satisfying bootloader and/or package script expectations.
  • Same for the 32-bit image, with the same kernel packages names and suffixes: v6, v7, v7l and v8. But one quirk is that the linux-image-rpi-v8 packages are not contained within the armhf package list, only in the arm64 ones, which is not enabled by default on 32-bit userland.

@MichaIng MichaIng added this to the v8.24 milestone Oct 19, 2023
@Joulinar
Copy link
Collaborator

uhh does not sounds like a quick win. 🙄

@onurcoskun14
Copy link

Any updates?

@axxis-creator
Copy link

Same Question. :-) I buy a RPI-5 and want to install DietPi for LoxBerry.
When can we expect a version?

@gaurishhs
Copy link

any updates on this?

@MichaIng
Copy link
Owner

We will post any updates here when we have some. I was looking further into it. The migration of the partition and installation of the new kernel packages is not too hard. But there is another big problem: Our scripts, and probably RPi's own scripts/programs expect config.txt and cmdline.txt in /boot, while for booting, both must be located in /boot/firmware (with the new package set). This is solved in RPi OS via symlinks. The problem is now that sed -i naturally replaces symlinks with actual files, and we use it a lot to edit their contents. Hence we must replace every occurrence across our whole code with sed -i --follow-symlinks to avoid this. Not understandable why this is not the default since ages, since I cannot imagine any circumstance where one would want to have a symlink replaced with its target file, when using sed on a symlink. However, I can do this quickly with a single command for the whole repo with the use of ... sed 😃. But this needs to be carefully reviewed, since there are variances like sed -Ei and probably some cases or code comments where the replacement must not be applied.

Another problem or better question is what we do with /boot/dietpi and /boot/dietpi.txt. The first, I think is fine to stay on the ext4 partition in the first place, resp. being moved there as part of the migration. The dietpi.txt is however nice to be on the FAT partition, at least before first boot, so one can apply automation and pre-configure the image from Windows and macOS easily. We just added a trailing FAT partition to all other images to make it as easy there as on RPi, so it would be horrible if we now removed this possibility for RPi. Theoretically we could do it on RPi the same way we do on other SBCs: As part of the first boot rootfs expansion script, mount the FAT partition and copy over dietpi.txt, dietpi-wifi.txt and all other optional pre-config files to the rootfs, then unmount it again. This must of course only happen on those RPi images which have the FAT partition mounted to /boot/firmware already.

Btw, does someone of you guys have an RPi 5 already and can tell me the revision code? That way we can add support to our hardware detection script already and provide RPi 5 compatible images between releases, if I am not able to finish this until this Saturday (v8.24 release):

mawk '/^Revision/{print $3;exit}' /proc/cpuinfo

The 3rd and 2nd last characters define the model. "14" was CM4, so it is probably "15" or "16".

@jboots07
Copy link

C04170

@adrianog91
Copy link

adrianog91 commented Nov 16, 2023 via email

@MichaIng
Copy link
Owner

Thanks. Found it now here as well: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#new-style-revision-codes

So "15" is the CM4S, which is a special industry customer variant, not publicly sold, and "16" for "internal use". Let's see whether "16" is a surprise, not announced yet 🙂.

@MichaIng
Copy link
Owner

First step: 6e848a1

@jboots07
Copy link

Thanks for your hard work and update! Let me know if I can help in any way, I have a pi 5.

@sorriso93
Copy link

I just received my PI5 and can't wait to install on it Dietpi to replace my PI4 :-)

@BugDave
Copy link

BugDave commented Nov 19, 2023

I just received mine too and am excited to put dietpi on it :-)

@MichaIng MichaIng modified the milestones: v8.24, v8.25 Nov 19, 2023
@m0jek
Copy link

m0jek commented Nov 21, 2023

@MichaIng when you are ready for testing in the wild on RaspberryPi 5, I am happy to test.

@FabioEight
Copy link

Available for testing on Pi 5 as well! Thanks for your efforts!

@elisenlebkuch
Copy link

Where can I get it?

@gaurishhs
Copy link

gaurishhs commented Nov 22, 2023

I don't think so there has been any image supporting Pi 5, You'll have to wait

@elisenlebkuch
Copy link

elisenlebkuch commented Nov 22, 2023

@gaurishhs
Copy link

I think that one is for Risc-V #6212

@LittleFreak
Copy link

LittleFreak commented Nov 26, 2023

Is it possible to add the dietpi later on a running raspbian?
I'll plan to docker all my programs anyway, so the host will stay pretty much raspbian lite until then.

If so, I'd just give my raspbian a diet later, when its compatible with raspi 5.

@MichaIng
Copy link
Owner

Is it possible to add the dietpi later on a running raspbian?

You mean on a Raspberry Pi OS? Currently does not work as dietpi-installer installs the old kernel/bootloader/firmware packages and our scripts use sed in various places a way that the /boot/config.txt => /boot/firmware/config.txt (same with cmdline.txt) symlinks are replaced with the actual files, so that the intended changes will not actually apply. See my longer comment above about this issue. Once we added compatibility, dietpi-installer can run on a new RPi OS image, but it will remove all installed software, including Docker in your case. What you could do is backup your /var/lib/docker. Then once dietpi-installer ran through and you boot DietPi, do not install Docker at first, but restore /var/lib/docker to /mnt/dietpi_userdata/docker-data, then install Docker via dietpi-software.

@cgmcfall
Copy link

Have a Pi5, happy to test :)

@alfredoanton82
Copy link

Also have a RPI5, happy to help testing.

@sergio-ingrao
Copy link

Here, I have another one ready to test.

@kris711
Copy link

kris711 commented May 24, 2024

grafik
After running the script on RPI3 I put SD Card to RPI 5 still I get the message
["Device-tree file "bcm2712-rpi-5-b.dtb" not found"] maybe I missed sth?
I am not sure if I understood it correctly but I do not have boot/firmware folder at all:
grafik

Thanks

@MichaIng
Copy link
Owner

@kris711 Next time (repeat), do not abort, but press ENTER to continue. Then you have a Debian Bookworm system. Next you need to run the migration script from first post, to actually migrate to the RPi kernel/firmware stack and install the RPi 5 kernel.

@Just2Confuse
Copy link

Im pretty sure this is not a Pi5 Problem? Or? DietPi (latest) was unexpected slower on a SSD, that all "writes".
#7092

@Joulinar
Copy link
Collaborator

@Just2Confuse pls have a look to your linked topic. There you got some questions.

@lukaszsobala
Copy link
Contributor

I've tested the in-place upgrade script on rpi zero 2w and on a 4B. In both cases it went smoothly and with no issues.

BTW, you can also create an image of the boot drive straight from the running system. Just stop any services and use dd to copy the drive to a file (you can also pipe the dd output to xz to have a smaller image).

@MichaIng MichaIng modified the milestones: v9.5, v9.6 Jun 10, 2024
@MichaIng
Copy link
Owner

BTW, you can also create an image of the boot drive straight from the running system. Just stop any services and use dd to copy the drive to a file (you can also pipe the dd output to xz to have a smaller image).

At best, remount the filesystems R/O:

dietpi-services stop
swapoff -a
mount -o remount,ro /boot
mount -o remount,ro /

It sometimes keep complaining "drive is busy", in which case one would need to further check which process is keeping a file open in writable mode, but this is safest to avoid inconsistencies.

Or use dietpi-imager from another Debian-based Linux system 🧐:

bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/DietPi/master/.build/images/dietpi-imager')

This has the benefit, that the resulting image file does not have the size of the root drive, but filesystem and partition are minimised at first (and automatically expanded again on reboot), and the image further compressed. Would be actually an idea to allow running this on the origin system itself, where it could attempt the R/O remounts automatically. Of course, the image would then need to be stored to another drive.

@MichaIng MichaIng modified the milestones: v9.6, v9.7 Jul 10, 2024
@Trainax
Copy link

Trainax commented Jul 11, 2024

Hi everyone, sorry if this is a duplicate comment however I couldn't find what I am looking for. With summer approaching my Pi 5 with an official active cooler fan is running hotter than usual. When I log in using SSH the banner says something like:

CPU temp : 56 °C / 132 °F : Running warm, but safe

Is there any way to change the PWM fan profile to make it spin faster and keep the CPU temperature lower? I know it says "safe", however I don't see any reason why it shouldn't be allowed to run cooler if possibile. Inspired by this GitHub Gist I was able to make the fan spin faster by writing a higher number to /sys/class/thermal/cooling_device0/cur_state however it appears to be overwritten every few seconds so the change isn't permanent. I would really like to be able to configure the PWM profile to have something like this:

CPU temp (°C) cur_state Fan speed
> 65 4 FULL
55 - 65 3 HIGH
45 - 54 2 MEDIUM
35 - 44 1 LOW
< 35 0 OFF

Thank you and sorry again if this is duplicate

@MichaIng MichaIng modified the milestones: v9.7, v9.8 Aug 26, 2024
@Joulinar
Copy link
Collaborator

Joulinar commented Sep 3, 2024

@MichaIng I just tested our migration script on a fresh empty RPi4 64bit installation without success.

During package install, I'm getting following error

Unpacking raspinfo (20240402-4) ...
Errors were encountered while processing:
 /tmp/apt-dpkg-install-SB1G7f/00-raspi-utils-core_20240402-4_arm64.deb
 /tmp/apt-dpkg-install-SB1G7f/07-raspi-utils-dt_20240402-4_arm64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
[FAILED] DietPi-Set_software | APT install linux-image-rpi-v8 raspi-firmware raspi-utils tiny-initramfs
 - Command: apt-get -y --allow-change-held-packages install linux-image-rpi-v8 raspi-firmware raspi-utils tiny-initramfs

On a retry some more information

[ INFO ] DietPi-Set_software | APT install linux-image-rpi-v8 raspi-firmware raspi-utils tiny-initramfs, please wait...
Reading package lists...
Building dependency tree...
Reading state information...
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 linux-image-rpi-v8 : Depends: linux-image-6.6.31+rpt-rpi-v8 (= 1:6.6.31-1+rpt1) but it is not going to be installed
 raspi-firmware : Depends: rsync but it is not going to be installed
 raspi-utils : Depends: raspi-utils-core but it is not going to be installed
               Depends: raspi-utils-dt but it is not going to be installed
               Breaks: libraspberrypi-bin (<= 1:2+git20231018~131943+3c97f76-1)
 raspinfo : Depends: raspi-utils-core but it is not going to be installed
 tiny-initramfs : Depends: tiny-initramfs-core (>= 0.1-5) but it is not going to be installed
                  Depends: tiny-initramfs-core (< 0.1-5+b+~) but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
[FAILED] DietPi-Set_software | APT install linux-image-rpi-v8 raspi-firmware raspi-utils tiny-initramfs
 - Command: apt-get -y --allow-change-held-packages install linux-image-rpi-v8 raspi-firmware raspi-utils tiny-initramfs

And running apt --fix-broken install give following

Preparing to unpack .../raspi-utils-core_20240402-4_arm64.deb ...
Unpacking raspi-utils-core (20240402-4) ...
dpkg: error processing archive /var/cache/apt/archives/raspi-utils-core_20240402-4_arm64.deb (--unpack):
 trying to overwrite '/usr/bin/vcgencmd', which is also in package libraspberrypi-bin 1:2+git20231018~131943+3c97f76-1
Errors were encountered while processing:
 /var/cache/apt/archives/raspi-utils-core_20240402-4_arm64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@DietPi4:~#

Means, we would need to have a look 🙄

@MichaIng
Copy link
Owner

MichaIng commented Sep 3, 2024

They did split raspi-utils in several sub packages. Maybe the declared conflict with libraspberrypi-bin was lost. I just pushed an update to our installer and migration script to install raspi-utils-core only, which is awesome as it pulls a lot less dependencies, but contains basically vcgencmd only. And indeed that one has no break/conflict with libraspberrypi-bin, but only raspi-utils has (which does not make sense since it is a dummy package). I'll report this to RPi devs.

@MichaIng
Copy link
Owner

MichaIng commented Sep 3, 2024

Done: RPi-Distro/repo#376

@MichaIng
Copy link
Owner

MichaIng commented Sep 12, 2024

The migration script should now work again.
EDIT: Yes, it works fine now 🙂.

@user2745
Copy link

Hey y'all, I hope this isn't an off topic question:

Is the img (available on the download page) ready for use in prod?

Thanks

@abusr
Copy link

abusr commented Sep 24, 2024

Is it not possible to have a single image for both raspi4 and raspi5, which will be containing both sbcs' dtbs (or whatever else detail) and their kernels together? So that at boot the user can select/set_default which path to boot, the 4 path or the 5 path. I might be asking something impossible because I don't know their boot chain at all but if it is possible, then ynot?

Because I am considering mixed environments with 4s or 5s and because this 'migration script' appears a one way solution, correct? Once one 'migrates' to 5, (s)he boot this installation in 4.

Since both 4 and 5 are well alive, flexibility could be appreciated here.

@Joulinar
Copy link
Collaborator

We have introduced an option especially for the RPi5 that allows you to select different kernel modules to ensure compatibility of the running system between RPi5 and other RPi SBCs. This should make it possible to use the SD card from an RPi5 in a RPi4 as well.

@MichaIng
Copy link
Owner

MichaIng commented Oct 7, 2024

Additionally, the "RPi234" image from here boots on RPi 5 as well. It contains the needed device tree, the kernel is just 4k page size instead of 16k, which is maybe a little performance penalty, but on the other hand is able to run 32-bit binaries (which the 16k page size kernel cannot).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment