RootFS is Read-only after failed Unbound installation

[moe93]

Details:

• Date | Thu Mar 11 14:36:00 EST 2021
• Bug report | a10a3039-e6c5-4ae7-999a-d48fa013a47f
• DietPi version | v7.0.2 (MichaIng/master)
• Image creator | DietPi Core Team
• Pre-image | Raspbian Lite
• Hardware | RPi 4 Model B (armv7l) (ID=4)
• Kernel version | Linux DietPi 5.10.17-v7l+ DietPi-Config | Minor NTP enhancements #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
• Distro | buster (ID=5,RASPBIAN=1)
• Command | mount -v -o rw,remount /
• Exit code | 32
• Software title | DietPi-Drive_Manager

Steps to reproduce:

1. Update image from 6.34.x to 7.x
2. Attempt to install Unbound on existing PiHole and PiVPN installation
3. Unbound install fails and prompts user to exit
4. Exit installation and problem occurred

Expected behaviour:

• Unbound should have installed properly. If it failed, it should have had a clean exit.

Actual behaviour:

• After Unbound failed to install, my SD became "corrupted" for the lack of a better term. I tried troubleshooting the issue on my own by remounting / as a RW but that failed. Tried using dietpi-drive_manager and that also failed. I can not use apt, I can not access dietpi-config, I can't use dpkg, and I can't even seem to reformat the SD card for a clean installation.

Extra details:

• ...

Additional logs:

root@DietPi:~# mount -o remount,rw /
mount: /: can't read superblock on /dev/mmcblk0p2.
root@DietPi:~# dietpi-drive_manager
[ INFO ] DietPi-Drive_Manager | Detecting drives, please wait...
[ INFO ] DietPi-Drive_Manager |  - Detected mounted drive: /dev/mmcblk0p2 > /
stty: 'standard input': Inappropriate ioctl for device
/boot/dietpi/func/dietpi-globals: line 499: ((: <= 120 : syntax error: operand e                                            xpected (error token is "<= 120 ")
/boot/dietpi/func/dietpi-globals: line 505: ((: >= 7 : syntax error: operand exp                                            ected (error token is ">= 7 ")
[FAILED] DietPi-Drive_Manager | mount -v -o rw,remount /
---------------------------------------------------------------------
[FAILED] DietPi-Drive_Manager | Unable to continue, DietPi-Drive_Manager will now terminate.

root@DietPi:~# dietpi-config
[FAILED] DietPi-Config | RootFS is currently Read Only (R/O) mounted. Aborting...
[ INFO ] DietPi-Config | DietPi requires RootFS to be Read/Write (R/W) mounted. Please run "dietpi-drive_manager" to re-enable.

[ravenclaw900]

Hello,
You can run a file system corruption check on reboot:

> /forcefsck
reboot
# After reboot, check result
journalctl -t systemd-fsck

[moe93]

Issuing the command failed with the following output

root@DietPi:~# > /forcefsck
-bash: /forcefsck: Read-only file system

[Joulinar]

usually this indicated some file system corruption. Can you check following dosfsck /dev/mmcblk0p1

[moe93]

The ouput is as follow:

root@DietPi:~# dosfsck /dev/mmcblk0p1
fsck.fat 4.1 (2017-01-24)
0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
1) Remove dirty bit
2) No action
?

EDIT: I don't understand what a dirty bit is exactly so I am not sure how to proceed

[ravenclaw900]

The dirty bit is set when mounted and unset when unmounted. Since you haven't unmounted the filesystem, it is still there. It is probably safe to remove, but there is no reason to.

[moe93]

Thank you for elaborating. However, would removing it resolve the Read-only issue I am having or would it have no effect?

[ravenclaw900]

Chances are it would have no effect, so I would recommend skipping it and seeing what fsck has to say next.

[moe93]

root@DietPi:~$ fsck /dev/mmcblk0p2
fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
rootfs: recovering journal
fsck.ext4: unable to set superblock flags on rootfs


rootfs: ********** WARNING: Filesystem still has errors **********

EDIT: Is it safe to assume my SD card has bitten the dust? It seems to me that it is a hardware issue at this point rather than software.

[Joulinar]

Yes looks like physical issues on the SD card. Usually @MichaIng has some good ideas on such issues

[MichaIng]

A filesystem error/corruption can be a hint for a dying SD card, but not necessarily. First I'd try to fix it and only if it re-appears without a previous system crash or such, I'd try a new one.

Although if R/W remount and fsck both fail in the first place, this indeed looks like a dying SD card: https://unix.stackexchange.com/questions/327863/fsck-wont-fsck-unable-to-set-superblock-flags

mount -o remount,rw / still does not work, right?

Please try to plug the SD card into an external Linux system and run fsck there.

[moe93]

I had to leave town, won’t be back until Sunday. I will try your suggestion and update you here once I get the opportunity.
Until then, have a great day and enjoy your weekend.

[moe93]

So I attempted running fsck on an external Linux system and got the same error regarding being unable to set the superblocks flag. I guess the SD card is indeed dead.

[MichaIng]

There is a procedure to make fsck use and restore an alternative/backup superblock: https://linuxexpresso.wordpress.com/2010/03/31/repair-a-broken-ext4-superblock-in-ubuntu/

Is it enough so that you can follow this?

[moe93]

Thank you for the link, but I had already bought a new SD card and got rid of the one giving me the read-only issue prior to reading the latest comment.
I appreciate all the time and responses you have provided me.

[Joulinar]

means, this can be closed now? Was Unbound installation working this time?

[moe93]

I have not had time to setup DietPi on the new SD due to work and time constraints so I can not comment on Unbound. However, I believe I will be holding off on the Unbound installation until the next DietPi update out of caution.
As for the issue ticket, yes, it can be closed.

Thank you.

[MichaIng]

The Unbound installation surely was not the reason, but the trigger for writes to bad blocks on the old SD card or similar. Means you don't need to worry that the install will corrupt your new SD card or so 😉. And it was not even necessarily a bad block, so the attempt to recover the superblock generally still makes sense, with the possibility to restore all other data untouched. But year, it's all a matter of weighing up whether it's worth the invested time and effort, when not being sure whether the same issue is hit again soon with that SD card.

I'll mark the issue as closed, feel free to reopen if required.