Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quartz64 | System boots read-only on first boot #6149

Closed
belveder79 opened this issue Feb 11, 2023 · 15 comments · Fixed by #6167
Closed

Quartz64 | System boots read-only on first boot #6149

belveder79 opened this issue Feb 11, 2023 · 15 comments · Fixed by #6167

Comments

@belveder79
Copy link

Creating a bug report/issue

Probably I'm making a stupid mistake, but current images for download don't work on Quartz64 A - there is no FAT partition so booting fails...

Required Information

  • DietPi version | Bookworm or Bullseye current image
  • Distro version | both
  • Kernel version | not known
  • SBC model | Quartz64 A
  • Power supply used 12V 1.8A
  • SD card used | internal emmc 32GB

Additional Information (if applicable)

Steps to reproduce

  1. Just download image,
  2. flash with Balena Etcher on MacPro M1, no FAT partition on written EMMC
  3. fails on first boot, can't find /boot and any files on it...

Expected behaviour

  1. should expand image, should have FAT partition, should basically work like on RPI3,4,RockPro64???

Actual behaviour

Extra details

Forum Question

@Joulinar
Copy link
Collaborator

There is no need to post your question twice on different platforms. This will not speedup anything. It will just double our efforts to maintain both issues. Pls try to stick to one platform in future. Thx for understanding.

It is fully correct to have a single ext4 partition for Quartz64 image. This is the current layout. There is no vfat partition anymore.

@MichaIng
Copy link
Owner

Coincidentally I'm currently testing some other kernel builds on Quartz64 A, B and SOQuartz, and the image, downloaded a few hours ago, boots fine here.

If it does not boot in your case, we'd need some more info, logs from serial console at best, else what you see on screen.

Also do you mean "internal" eMMC? How did you write to it? I have an eMMC slot here, which works, but I cannot test whether it works from a soldered/internal eMMC.

@belveder79
Copy link
Author

ok, sorry for adding the duplicate...

I just downloaded it and flashed it with Balena Etcher - like I did a hundred times before... it is not a soldered emmc, it is written with a usb writer (I tried with two different ones, same result).

I end up with the root login, but upon login it confronts me with /boot/dietpi.txt missing and does not move on with the setup process...

I have no WIFI dongle on the Quartz64 A, just connected over Ethernet, but it seems to not come up... I remember seeing a lot of red-flagged "Failure" messages during boot - have no access right now....

@MichaIng
Copy link
Owner

MichaIng commented Feb 11, 2023

it confronts me with /boot/dietpi.txt missing

That is strange indeed, since the file is shipped with the image. Would be unexpected but probably the eMMC chip is dying? Did you try it with an SD card? And did you try to inspect the eMMC from another system, right after flashing it, verifying that /boot/dietpi.txt is there, on the only partition and filesystem on this image? Also is there anything else in /boot?

ls -al /boot

Obviously extlinux, kernel and device tree, otherwise it wouldn't boot 🤔. Strange that a single particular file is missing, or probably some others as well which we didn't recognise yet.

@belveder79
Copy link
Author

Ok, so both images actually work fine, but there's a catch - it NEVER works on first reboot, but only on the second one.
I repeated the flashing/booting 4 times now and it was the same every time. The root partition is mounted readonly, so the most I could get out was this screen shot after the first boot:

image

It seems that there's a problem with the checksum after expansion. However, pressing reset and just letting it boot normally brings it up fine!

@MichaIng
Copy link
Owner

Can you show the output of:

cat /var/tmp/dietpi/logs/fs_partition_resize.log

Probably there is a bad block (one that cannot be written to) at the location where one of the backup superblocks is written to. This is always the same location for same filesystem sizes. Since its only a backup superbock, not such a problem. What happens if you run fsck on reboot?

> /forcefsck

After reboot

journalctl -t systemd-fsck

@belveder79
Copy link
Author

The first one says

Removed "/etc/systemd/system/local-fs.target.wants/dietpi-fs_partition_resize.service".

The second one runs fine actually:

root@DietPi:~# journalctl -t systemd-fsck
Feb 12 21:02:54 DietPi systemd-fsck[216]: Please pass 'fsck.mode=force' on the kernel command line rather than creating /forcefsck on the root file system.
Feb 12 21:02:54 DietPi systemd-fsck[221]: e2fsck 1.46.6 (1-Feb-2023)
Feb 12 21:02:54 DietPi systemd-fsck[221]: Pass 1: Checking inodes, blocks, and sizes
Feb 12 21:02:55 DietPi systemd-fsck[221]: Pass 2: Checking directory structure
Feb 12 21:02:57 DietPi systemd-fsck[221]: Pass 3: Checking directory connectivity
Feb 12 21:02:57 DietPi systemd-fsck[221]: Pass 4: Checking reference counts
Feb 12 21:02:57 DietPi systemd-fsck[221]: Pass 5: Checking group summary information
Feb 12 21:02:57 DietPi systemd-fsck[221]: /dev/mmcblk1p1: 142594/1875184 files (0.1% non-contiguous), 1874948/7629824 blocks

@MichaIng
Copy link
Owner

Hmm, filesystem integrity seems fine (now). The resize script log seems to be cut due to the read-only remount and some more expected log files likely were still in buffer at that time.

As said, likely an issue with the eMMC. Generally, I recommend to check

dmesg -l 0,1,2,3

by times to check for kernel errors, which would include filesystem and raw I/O errors. Probably check that after you did larger software installs or upgrades which imply writing a lot of new data.

@belveder79
Copy link
Author

I will do another install with an other emmc, see what happens... will report back...

@belveder79
Copy link
Author

same thing... different Quartz board, different EMMC... first boot ends up in login with R/O file system, reboot works fine...

@MichaIng
Copy link
Owner

Can you test with the new image here: https://dietpi.com/downloads/images/testing/

@belveder79
Copy link
Author

same...

@MichaIng MichaIng changed the title Images for Quartz64 A don't work (neither Bookworm, nor Bullseye) Quartz64 | System boots read-only on first boot Feb 13, 2023
@MichaIng MichaIng added this to the v8.15 milestone Feb 13, 2023
@MichaIng
Copy link
Owner

MichaIng commented Feb 13, 2023

I can replicate it now. Very strange since those images are created the exact same way than for any other SBCs, with common single ext4 partition MBR layout. Also, expanding the filesystem from an external system does not lead to this CRC check failure, also not when doing online resizing. I feat it is a bug in this particular kernel version.

When checking logs before doing the reboot, it reveals some more infos, that "the filesystem was found in read-only mode while checking for online resizing support" or so. Most of the log lines are lost after reboot, I guess they were accessible from but stuck in filesystem cache before reboot. This indicates that that read-only mode is there before doing the resize. dmesg however shows it clearly the other way round, as can be seen in your screenshot, which matches mine exactly.

When doing fsck on the same boot session, no errors are found, but an optimisable extent tree. I'll check now whether optimising this makes any difference, but I doubt. (EDIT: It does not) I'll do some more debugging, and as last resort add a check for R/O mount after resizing, and doing a reboot in case.

MichaIng added a commit that referenced this issue Feb 13, 2023
- DietPi-Installer | Use rng-tools5 on NanoPi M4/T4/NEO4 but haveged on Quartz64
- DietPi-Set_software | Fix serial getty service override on Bullseye and below, where using a dash as console device somehow fails
- DietPi-FS_partition_resize | reboot if resizing fails on ext4. This is mainly to resolve the issue on Quartz64 where for unknown reasons resizing leads to a CRC error in first superblock backup, leading to read-only remount, while actually nothing bad happened, fsck does not report any error and after a reboot all works as expected: #6149
MichaIng added a commit that referenced this issue Feb 13, 2023
- DietPi-Installer | Temporary workaround for read-only mount after filesystem expansion on Quartz64 until v8.15 has been released: #6149
@MichaIng MichaIng added Solution available 🥂 Definite solution has been done and removed Investigating 🤔 labels Feb 13, 2023
@MichaIng
Copy link
Owner

MichaIng commented Feb 13, 2023

Solved, new images are available already (above link) and will be moved to stable downloads when the last two (not Quartz64) have finished.

EDIT: With the reboot on resize failure more a workaround than a real solution. Probably with latest Linux v6.1.11 this will be solved: #6149

@MichaIng MichaIng added Workaround available 🆗 Workaround is available/has been implemented, but a definite solution should be found when possible. and removed Solution available 🥂 Definite solution has been done labels Feb 15, 2023
@MichaIng MichaIng linked a pull request Feb 18, 2023 that will close this issue
@MichaIng MichaIng removed the Workaround available 🆗 Workaround is available/has been implemented, but a definite solution should be found when possible. label Feb 19, 2023
@MichaIng MichaIng added the Solution available 🥂 Definite solution has been done label Feb 19, 2023
@MichaIng
Copy link
Owner

With the new/stable kernel builds the resizing works now reliably on first boot: https://dietpi.com/downloads/images/testing/
I'll leave the workaround (reboot on failure) in place, since it is generally not a bad idea to be failsafe.

Kernel can be upgraded with the firmware package from here: https://dietpi.com/downloads/binaries/
or will be offered on next DietPi update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants