Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DietPi Failing to Update #3048

Closed
tenryou-17 opened this issue Aug 10, 2019 · 11 comments
Closed

DietPi Failing to Update #3048

tenryou-17 opened this issue Aug 10, 2019 · 11 comments

Comments

@tenryou-17
Copy link

Details:

  • Date | Sat 10 Aug 20:26:57 BST 2019
  • Bug report | N/A
  • DietPi version | v6.24.1 (MichaIng/master)
  • Img creator | DietPi Core Team
  • Pre-image | Raspbian Lite
  • SBC device | RPi 3 Model B (armv7l) (index=3)
  • Kernel version | General | Update VM images to Stretch #1219 SMP Tue May 14 21:20:58 BST 2019
  • Distro | stretch (index=4)
  • Command | G_AGUG
  • Exit code | 100
  • Software title | DietPi-Update

Steps to reproduce:

  1. ran "dietpi-update"

Expected behaviour:

  • dietpi should update

Actual behaviour:

  • it fails to update. see text in additional logs.

Extra details:

  • I have ran the update with and without my dietpi being behind a VPN
  • I have also changed the APT and NTP mirrors and ran them both without the dietpi being behind a VPN, with no luck.

Additional logs:

Log file contents:
Preconfiguring packages ...
dpkg: warning: files list file for package 'libnpth0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'console-setup' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libcairo2:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libnettle6:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'keyboard-configuration' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libidn2-0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libip4tc0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'raspi-copies-and-fills' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libxext6:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libxfixes3:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'sudo' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libreadline7:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'multiarch-support' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'systemd-sysv' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libsoxr0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'unzip' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libx11-6:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libdns-export162' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'wireless-regdb' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libicu57:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'tar' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libssl1.0.2:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'sensible-utils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libssh2-1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'init-system-helpers' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ncurses-base' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libqt5xml5:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libssl1.1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libbluray1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libkrb5-3:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libtext-charwidth-perl' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libgme0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ethtool' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libnewt0.52:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'htop' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'console-setup-linux' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'gnupg-agent' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'cron' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libsepol1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libidn11:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libgcc1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'netbase' missing; assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting:
 files list file for package 'dash' is missing final newline
E: Sub-process /usr/bin/dpkg returned an error code (2)
@MichaIng
Copy link
Owner

MichaIng commented Aug 10, 2019 •
edited
Loading

@tenryou-17
Many thanks for your report.

Hmm something with the package file lists is damaged.
Can you paste:

grep -r 'admindir' /etc/dpkg
ls -lh /var/lib/dpkg/info/*.list

@tenryou-17
Copy link
Author

@MichaIng
Thanks for the reply! Copied and pasted each line. Still an error. Here is the output with the VPN disabled, to reduce the number of variables it could be.

Log file contents:
Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Hit:2 https://archive.raspberrypi.org/debian stretch InRelease
Reading package lists...
E: Problem parsing dependency 20
E: Error occurred while processing twinkle (NewVersion2)
E: Problem with MergeList /var/lib/apt/lists/raspbian.raspberrypi.org_raspbian_dists_stretch_main_binary-armhf_Packages
E: The package lists or status file could not be parsed or opened.

@MichaIng
Copy link
Owner

MichaIng commented Aug 10, 2019 •
edited
Loading

@tenryou-17
Please paste the output of the two commands I posted above here. Those were not to fix anything but to investigate the issue 😉.

@tenryou-17
Copy link
Author

Ha! My bad. There was no output for the first line. Here is the output for the second line:

-rw-r--r-- 1 root root 6.3K May 20 00:59 /var/lib/dpkg/info/adduser.list
-rw-r--r-- 1 root root 9.9K May 20 00:59 /var/lib/dpkg/info/apt.list
-rw-r--r-- 1 root root  329 Mar  4 12:57 /var/lib/dpkg/info/apt-transport-https.list
-rw-r--r-- 1 root root 6.2K Mar  4 12:57 /var/lib/dpkg/info/apt-utils.list
-rw-r--r-- 1 root root 1.8K May 20 00:59 /var/lib/dpkg/info/base-files.list
-rw-r--r-- 1 root root 1.1K May 20 03:36 /var/lib/dpkg/info/base-passwd.list
-rw-r--r-- 1 root root  27K Mar  4 12:57 /var/lib/dpkg/info/bash-completion.list
-rw-r--r-- 1 root root 4.5K Mar  4 12:57 /var/lib/dpkg/info/bash.list
-rw-r--r-- 1 root root  553 May 20 03:36 /var/lib/dpkg/info/bc.list
-rw-r--r-- 1 root root  801 Mar  4 12:57 /var/lib/dpkg/info/bsdutils.list
-rw-r--r-- 1 root root  689 Mar  4 12:57 /var/lib/dpkg/info/bzip2.list
-rw-r--r-- 1 root root  12K May 20 00:59 /var/lib/dpkg/info/ca-certificates.list
-rw-r--r-- 1 root root 1.2K May 20 00:59 /var/lib/dpkg/info/cbon.list
-rw-r--r-- 1 root root 9.5K May 20 00:59 /var/lib/dpkg/info/coreutils.list
-rw-r--r-- 1 root root  743 May 20 00:59 /var/lib/dpkg/info/crda.list
-rw-r--r-- 1 root root  319 Mar  4 12:57 /var/lib/dpkg/info/curl.list
-rw-r--r-- 2 root root 5.4K Aug 20  2018 /var/lib/dpkg/info/dash.list
-rw-r--r-- 1 root root 7.5K May 20 00:59 /var/lib/dpkg/info/debconf.list
-rw-r--r-- 1 root root 3.0K May 20 00:59 /var/lib/dpkg/info/debianutils.list
-rw-r--r-- 1 root root  787 May 20 03:36 /var/lib/dpkg/info/device-tree-compiler.list
-rw-r--r-- 1 root root 3.6K Mar  4 12:57 /var/lib/dpkg/info/diffutils.list
-rw-r--r-- 1 root root  804 May 20 00:59 /var/lib/dpkg/info/dirmngr.list
-rw-r--r-- 1 root root  579 May 20 00:59 /var/lib/dpkg/info/dmsetup.list
-rw-r--r-- 1 root root 1.1K May 20 00:59 /var/lib/dpkg/info/dosfstools.list
-rw-r--r-- 1 root root 8.2K May 20 00:59 /var/lib/dpkg/info/dpkg.list
-rw-r--r-- 1 root root  615 May 20 00:59 /var/lib/dpkg/info/dropbear-bin.list
-rw-r--r-- 1 root root  348 Mar  4 12:57 /var/lib/dpkg/info/dropbear-run.list
-rw-r--r-- 1 root root  309 Mar  4 12:57 /var/lib/dpkg/info/e2fslibs:armhf.list
-rw-r--r-- 1 root root 3.6K May 20 00:59 /var/lib/dpkg/info/e2fsprogs.list
-rw-r--r-- 1 root root  396 May 20 00:59 /var/lib/dpkg/info/fake-hwclock.list
-rw-r--r-- 1 root root 4.4K May 20 03:36 /var/lib/dpkg/info/findutils.list
-rw-r--r-- 1 root root  12K Mar  4 12:57 /var/lib/dpkg/info/firmware-amd-graphics.list
-rw-r--r-- 1 root root 5.4K Mar  4 12:57 /var/lib/dpkg/info/firmware-atheros.list
-rw-r--r-- 1 root root 1.6K Mar  4 12:57 /var/lib/dpkg/info/firmware-brcm80211.list
-rw-r--r-- 1 root root 2.6K Mar  4 12:57 /var/lib/dpkg/info/firmware-iwlwifi.list
-rw-r--r-- 1 root root  163 Mar  4 12:57 /var/lib/dpkg/info/firmware-linux-nonfree.list
-rw-r--r-- 1 root root 9.0K Mar  4 12:57 /var/lib/dpkg/info/firmware-misc-nonfree.list
-rw-r--r-- 1 root root 2.5K Mar  4 12:57 /var/lib/dpkg/info/firmware-realtek.list
-rw-r--r-- 1 root root 2.7K May 20 03:35 /var/lib/dpkg/info/fontconfig-config.list
-rw-r--r-- 1 root root 2.1K May 20 03:35 /var/lib/dpkg/info/fonts-dejavu-core.list
-rw-r--r-- 1 root root  397 Mar  4 12:57 /var/lib/dpkg/info/gcc-4.6-base:armhf.list
-rw-r--r-- 1 root root  355 Mar  4 12:57 /var/lib/dpkg/info/gcc-4.7-base:armhf.list
-rw-r--r-- 1 root root  355 Mar  4 12:57 /var/lib/dpkg/info/gcc-4.8-base:armhf.list
-rw-r--r-- 1 root root  355 Mar  4 12:57 /var/lib/dpkg/info/gcc-4.9-base:armhf.list
-rw-r--r-- 1 root root  343 Mar  4 12:57 /var/lib/dpkg/info/gcc-5-base:armhf.list
-rw-r--r-- 1 root root  343 Mar  4 12:57 /var/lib/dpkg/info/gcc-6-base:armhf.list
-rw-r--r-- 1 root root 5.3K Mar  4 12:57 /var/lib/dpkg/info/gettext-base.list
-rw-r--r-- 1 root root 1.5K May 20 00:59 /var/lib/dpkg/info/gnupg.list
-rw-r--r-- 1 root root  230 Mar  4 12:57 /var/lib/dpkg/info/gpgv.list
-rw-r--r-- 1 root root 4.6K Mar  4 12:57 /var/lib/dpkg/info/grep.list
-rw-r--r-- 1 root root  928 Mar  4 12:57 /var/lib/dpkg/info/gzip.list
-rw-r--r-- 1 root root  996 May 20 00:59 /var/lib/dpkg/info/hdparm.list
-rw-r--r-- 1 root root  391 Mar  4 12:57 /var/lib/dpkg/info/hostname.list
-rw-r--r-- 1 root root 1.7K May 20 00:59 /var/lib/dpkg/info/ifupdown.list
-rw-r--r-- 1 root root  102 Mar  4 12:57 /var/lib/dpkg/info/init.list
-rw-r--r-- 1 root root  598 May 20 00:59 /var/lib/dpkg/info/install-info.list
-rw-r--r-- 1 root root 4.1K May 20 00:59 /var/lib/dpkg/info/iproute2.list
-rw-r--r-- 1 root root  435 May 20 00:59 /var/lib/dpkg/info/iputils-ping.list
-rw-r--r-- 1 root root  753 May 20 00:59 /var/lib/dpkg/info/isc-dhcp-client.list
-rw-r--r-- 1 root root  178 May 20 00:59 /var/lib/dpkg/info/iw.list
-rw-r--r-- 1 root root 5.2K May 20 03:36 /var/lib/dpkg/info/kbd.list
-rw-r--r-- 1 root root 1023 May 20 00:59 /var/lib/dpkg/info/kmod.list
-rw-r--r-- 1 root root  262 Mar  4 12:57 /var/lib/dpkg/info/libacl1:armhf.list
-rw-r--r-- 1 root root  341 Mar  4 12:57 /var/lib/dpkg/info/libapparmor1:armhf.list
-rw-r--r-- 1 root root 4.8K Mar  4 12:57 /var/lib/dpkg/info/libapt-inst2.0:armhf.list
-rw-r--r-- 1 root root 4.8K Mar  4 12:57 /var/lib/dpkg/info/libapt-pkg5.0:armhf.list
-rw-r--r-- 1 root root  296 Mar  4 12:57 /var/lib/dpkg/info/libassuan0:armhf.list
-rw-r--r-- 1 root root  268 Mar  4 12:57 /var/lib/dpkg/info/libattr1:armhf.list
-rw-r--r-- 1 root root  274 Mar  4 12:57 /var/lib/dpkg/info/libaudit1:armhf.list
-rw-r--r-- 1 root root  249 May 20 00:59 /var/lib/dpkg/info/libaudit-common.list
-rw-r--r-- 1 root root  416 May 20 03:36 /var/lib/dpkg/info/libavcodec57:armhf.list
-rw-r--r-- 1 root root  423 May 20 03:36 /var/lib/dpkg/info/libavformat57:armhf.list
-rw-r--r-- 1 root root  320 May 20 03:35 /var/lib/dpkg/info/libavutil55:armhf.list
-rw-r--r-- 1 root root  274 Mar  4 12:57 /var/lib/dpkg/info/libblkid1:armhf.list
-rw-r--r-- 1 root root  367 May 20 03:37 /var/lib/dpkg/info/libboost-chrono1.61.0:armhf.list
-rw-r--r-- 1 root root  367 May 20 03:37 /var/lib/dpkg/info/libboost-random1.61.0:armhf.list
-rw-r--r-- 1 root root  367 May 20 03:37 /var/lib/dpkg/info/libboost-system1.61.0:armhf.list
-rw-r--r-- 1 root root  367 May 20 03:37 /var/lib/dpkg/info/libboost-system1.62.0:armhf.list
-rw-r--r-- 1 root root  277 May 20 03:35 /var/lib/dpkg/info/libbsd0:armhf.list
-rw-r--r-- 1 root root  313 Mar  4 12:57 /var/lib/dpkg/info/libbz2-1.0:armhf.list
-rw-r--r-- 1 root root  14K Mar  4 12:57 /var/lib/dpkg/info/libc6:armhf.list
-rw-r--r-- 1 root root  261 Mar  4 12:57 /var/lib/dpkg/info/libcap2:armhf.list
-rw-r--r-- 1 root root  562 May 20 00:59 /var/lib/dpkg/info/libcap2-bin.list
-rw-r--r-- 1 root root  289 Mar  4 12:57 /var/lib/dpkg/info/libcap-ng0:armhf.list
-rw-r--r-- 1 root root 1.3K Mar  4 12:57 /var/lib/dpkg/info/libc-bin.list
-rw-r--r-- 1 root root  341 May 20 03:36 /var/lib/dpkg/info/libchromaprint1:armhf.list
-rw-r--r-- 1 root root 3.5K Mar  4 12:57 /var/lib/dpkg/info/libc-l10n.list
-rw-r--r-- 1 root root  241 Mar  4 12:57 /var/lib/dpkg/info/libcomerr2:armhf.list
-rw-r--r-- 1 root root  349 Mar  4 12:57 /var/lib/dpkg/info/libcryptsetup4:armhf.list
-rw-r--r-- 1 root root  451 Mar  4 12:57 /var/lib/dpkg/info/libcurl3:armhf.list
-rw-r--r-- 1 root root  514 Mar  4 12:57 /var/lib/dpkg/info/libcurl3-gnutls:armhf.list
-rw-r--r-- 1 root root  337 Mar  4 12:57 /var/lib/dpkg/info/libdb5.3:armhf.list
-rw-r--r-- 1 root root  485 Mar  4 12:57 /var/lib/dpkg/info/libdbus-1-3:armhf.list
-rw-r--r-- 1 root root  285 Mar  4 12:57 /var/lib/dpkg/info/libdebconfclient0:armhf.list
-rw-r--r-- 1 root root  340 Mar  4 12:57 /var/lib/dpkg/info/libdevmapper1.02.1:armhf.list
-rw-r--r-- 1 root root  419 May 20 03:37 /var/lib/dpkg/info/libdouble-conversion1:armhf.list
-rw-r--r-- 1 root root  295 May 20 03:35 /var/lib/dpkg/info/libdrm2:armhf.list
-rw-r--r-- 1 root root 1.1K Mar  4 12:57 /var/lib/dpkg/info/libelf1:armhf.list
-rw-r--r-- 1 root root 3.0K May 20 03:35 /var/lib/dpkg/info/libexif12:armhf.list
-rw-r--r-- 1 root root  419 May 20 03:36 /var/lib/dpkg/info/libexpat1:armhf.list
-rw-r--r-- 1 root root  274 Mar  4 12:57 /var/lib/dpkg/info/libfdisk1:armhf.list
-rw-r--r-- 1 root root  242 Mar  4 12:57 /var/lib/dpkg/info/libffi6:armhf.list
-rw-r--r-- 1 root root  262 May 20 03:36 /var/lib/dpkg/info/libflac8:armhf.list
-rw-r--r-- 1 root root  413 May 20 03:36 /var/lib/dpkg/info/libfontconfig1:armhf.list
-rw-r--r-- 1 root root  636 May 20 03:36 /var/lib/dpkg/info/libfreetype6:armhf.list
-rw-r--r-- 1 root root  433 Mar  4 12:57 /var/lib/dpkg/info/libgcrypt20:armhf.list
-rw-r--r-- 1 root root 1.5K May 20 03:37 /var/lib/dpkg/info/libglib2.0-0:armhf.list
-rw-r--r-- 1 root root  285 Mar  4 12:57 /var/lib/dpkg/info/libgmp10:armhf.list
-rw-r--r-- 1 root root 2.1K Mar  4 12:57 /var/lib/dpkg/info/libgnutls30:armhf.list
-rw-r--r-- 1 root root  184 May 20 03:35 /var/lib/dpkg/info/libgomp1:armhf.list
-rw-r--r-- 1 root root 2.3K Mar  4 12:57 /var/lib/dpkg/info/libgpg-error0:armhf.list
-rw-r--r-- 1 root root  356 May 20 03:35 /var/lib/dpkg/info/libgsm1:armhf.list
-rw-r--r-- 1 root root  477 Mar  4 12:57 /var/lib/dpkg/info/libgssapi-krb5-2:armhf.list
-rw-r--r-- 1 root root  176 Mar  4 12:57 /var/lib/dpkg/info/libhogweed4:armhf.list
-rw-r--r-- 1 root root  409 May 20 03:36 /var/lib/dpkg/info/libid3tag0:armhf.list
-rw-r--r-- 1 root root  316 Mar  4 12:57 /var/lib/dpkg/info/libisc-export160.list
-rw-r--r-- 1 root root  185 Mar  4 12:57 /var/lib/dpkg/info/libiw30:armhf.list
-rw-r--r-- 1 root root  422 May 20 03:36 /var/lib/dpkg/info/libjpeg62-turbo:armhf.list
-rw-r--r-- 1 root root  308 Mar  4 12:57 /var/lib/dpkg/info/libk5crypto3:armhf.list
-rw-r--r-- 1 root root  249 Mar  4 12:57 /var/lib/dpkg/info/libkeyutils1:armhf.list
-rw-r--r-- 1 root root  268 Mar  4 12:57 /var/lib/dpkg/info/libkmod2:armhf.list
-rw-r--r-- 1 root root  326 Mar  4 12:57 /var/lib/dpkg/info/libkrb5support0:armhf.list
-rw-r--r-- 1 root root  380 Mar  4 12:57 /var/lib/dpkg/info/libksba8:armhf.list
-rw-r--r-- 1 root root  594 Mar  4 12:57 /var/lib/dpkg/info/libldap-2.4-2:armhf.list
-rw-r--r-- 1 root root  252 May 20 00:59 /var/lib/dpkg/info/libldap-common.list
-rw-r--r-- 1 root root  836 Mar  4 12:57 /var/lib/dpkg/info/liblocale-gettext-perl.list
-rw-r--r-- 1 root root  245 Mar  4 12:57 /var/lib/dpkg/info/liblz4-1:armhf.list
-rw-r--r-- 1 root root  363 Mar  4 12:57 /var/lib/dpkg/info/liblzma5:armhf.list
-rw-r--r-- 1 root root  612 May 20 03:34 /var/lib/dpkg/info/libmemcached11:armhf.list
-rw-r--r-- 1 root root  448 May 20 03:34 /var/lib/dpkg/info/libmemcachedutil2:armhf.list
-rw-r--r-- 1 root root  226 Mar  4 12:57 /var/lib/dpkg/info/libmnl0:armhf.list
-rw-r--r-- 1 root root  274 Mar  4 12:57 /var/lib/dpkg/info/libmount1:armhf.list
-rw-r--r-- 1 root root  317 May 20 03:35 /var/lib/dpkg/info/libmp3lame0:armhf.list
-rw-r--r-- 1 root root  316 May 20 03:36 /var/lib/dpkg/info/libmpg123-0:armhf.list
-rw-r--r-- 1 root root  458 Mar  4 12:57 /var/lib/dpkg/info/libncurses5:armhf.list
-rw-r--r-- 1 root root  467 Mar  4 12:57 /var/lib/dpkg/info/libncursesw5:armhf.list
-rw-r--r-- 1 root root  393 Mar  4 12:57 /var/lib/dpkg/info/libnghttp2-14:armhf.list
-rw-r--r-- 1 root root  385 Mar  4 12:57 /var/lib/dpkg/info/libnl-3-200:armhf.list
-rw-r--r-- 1 root root  315 Mar  4 12:57 /var/lib/dpkg/info/libnl-genl-3-200:armhf.list
-rw-r--r-- 1 root root  293 May 20 03:35 /var/lib/dpkg/info/libogg0:armhf.list
-rw-r--r-- 1 root root  321 May 20 03:35 /var/lib/dpkg/info/libopenjp2-7:armhf.list
-rw-r--r-- 1 root root  318 May 20 03:36 /var/lib/dpkg/info/libopenmpt0:armhf.list
-rw-r--r-- 1 root root  262 May 20 03:35 /var/lib/dpkg/info/libopus0:armhf.list
-rw-r--r-- 1 root root  394 Mar  4 12:57 /var/lib/dpkg/info/libp11-kit0:armhf.list
-rw-r--r-- 1 root root  716 Mar  4 12:57 /var/lib/dpkg/info/libpam0g:armhf.list
-rw-r--r-- 1 root root 4.6K May 20 00:59 /var/lib/dpkg/info/libpam-modules:armhf.list
-rw-r--r-- 1 root root  642 May 20 00:59 /var/lib/dpkg/info/libpam-modules-bin.list
-rw-r--r-- 1 root root 8.1K May 20 00:59 /var/lib/dpkg/info/libpam-runtime.list
-rw-r--r-- 1 root root  280 Mar  4 12:57 /var/lib/dpkg/info/libparted2:armhf.list
-rw-r--r-- 1 root root  316 May 20 03:37 /var/lib/dpkg/info/libpcre16-3:armhf.list
-rw-r--r-- 1 root root  618 Mar  4 12:57 /var/lib/dpkg/info/libpcre3:armhf.list
-rw-r--r-- 1 root root  308 Mar  4 12:57 /var/lib/dpkg/info/libpcsclite1:armhf.list
-rw-r--r-- 1 root root  419 May 20 03:36 /var/lib/dpkg/info/libpixman-1-0:armhf.list
-rw-r--r-- 1 root root  518 May 20 03:36 /var/lib/dpkg/info/libpng16-16:armhf.list
-rw-r--r-- 1 root root 3.3K Mar  4 12:57 /var/lib/dpkg/info/libpopt0:armhf.list
-rw-r--r-- 1 root root  321 Mar  4 12:57 /var/lib/dpkg/info/libprocps6:armhf.list
-rw-r--r-- 1 root root  348 May 20 03:37 /var/lib/dpkg/info/libproxy1v5:armhf.list
-rw-r--r-- 1 root root  278 Mar  4 12:57 /var/lib/dpkg/info/libpsl5:armhf.list
-rw-r--r-- 1 root root  505 May 20 03:37 /var/lib/dpkg/info/libqt5core5a:armhf.list
-rw-r--r-- 1 root root  499 May 20 03:37 /var/lib/dpkg/info/libqt5dbus5:armhf.list
-rw-r--r-- 1 root root  850 May 20 03:37 /var/lib/dpkg/info/libqt5network5:armhf.list
-rw-r--r-- 1 root root 1.9K May 20 03:27 /var/lib/dpkg/info/libraspberrypi0.list
-rw-r--r-- 1 root root 1.3K May 20 03:27 /var/lib/dpkg/info/libraspberrypi-bin.list
-rw-r--r-- 1 root root  580 Mar  4 12:57 /var/lib/dpkg/info/libreadline6:armhf.list
-rw-r--r-- 1 root root  238 Mar  4 12:57 /var/lib/dpkg/info/librtmp1:armhf.list
-rw-r--r-- 1 root root  442 Mar  4 12:57 /var/lib/dpkg/info/libsasl2-2:armhf.list
-rw-r--r-- 1 root root  478 Mar  4 12:57 /var/lib/dpkg/info/libsasl2-modules-db:armhf.list
-rw-r--r-- 1 root root  286 Mar  4 12:57 /var/lib/dpkg/info/libseccomp2:armhf.list
-rw-r--r-- 1 root root  241 Mar  4 12:57 /var/lib/dpkg/info/libselinux1:armhf.list
-rw-r--r-- 1 root root  258 Mar  4 12:57 /var/lib/dpkg/info/libsemanage1:armhf.list
-rw-r--r-- 1 root root  282 May 20 00:59 /var/lib/dpkg/info/libsemanage-common.list
-rw-r--r-- 1 root root  305 May 20 03:35 /var/lib/dpkg/info/libshine3:armhf.list
-rw-r--r-- 1 root root  505 Mar  4 12:57 /var/lib/dpkg/info/libslang2:armhf.list
-rw-r--r-- 1 root root  298 Mar  4 12:57 /var/lib/dpkg/info/libsmartcols1:armhf.list
-rw-r--r-- 1 root root  319 May 20 03:35 /var/lib/dpkg/info/libsnappy1v5:armhf.list
-rw-r--r-- 1 root root  267 May 20 03:35 /var/lib/dpkg/info/libspeex1:armhf.list
-rw-r--r-- 1 root root  394 Mar  4 12:57 /var/lib/dpkg/info/libsqlite3-0:armhf.list
-rw-r--r-- 1 root root  219 Mar  4 12:57 /var/lib/dpkg/info/libss2:armhf.list
-rw-r--r-- 1 root root  548 May 20 03:36 /var/lib/dpkg/info/libssh-gcrypt-4:armhf.list
-rw-r--r-- 1 root root  716 Mar  4 12:57 /var/lib/dpkg/info/libstdc++6:armhf.list
-rw-r--r-- 1 root root  429 May 20 03:36 /var/lib/dpkg/info/libswresample2:armhf.list
-rw-r--r-- 1 root root  262 May 20 00:59 /var/lib/dpkg/info/libsystemd0:armhf.list
-rw-r--r-- 1 root root  394 Mar  4 12:57 /var/lib/dpkg/info/libtasn1-6:armhf.list
-rw-r--r-- 1 root root  814 Mar  4 12:57 /var/lib/dpkg/info/libtext-iconv-perl.list
-rw-r--r-- 1 root root  389 Mar  4 12:57 /var/lib/dpkg/info/libtext-wrapi18n-perl.list
-rw-r--r-- 1 root root  578 May 20 03:36 /var/lib/dpkg/info/libtheora0:armhf.list
-rw-r--r-- 1 root root  460 Mar  4 12:57 /var/lib/dpkg/info/libtinfo5:armhf.list
-rw-r--r-- 1 root root  473 May 20 03:37 /var/lib/dpkg/info/libtorrent-rasterbar9.list
-rw-r--r-- 1 root root  317 May 20 03:36 /var/lib/dpkg/info/libtwolame0:armhf.list
-rw-r--r-- 1 root root  246 May 20 00:59 /var/lib/dpkg/info/libudev1:armhf.list
-rw-r--r-- 1 root root  494 Mar  4 12:57 /var/lib/dpkg/info/libunistring0:armhf.list
-rw-r--r-- 1 root root  325 Mar  4 12:57 /var/lib/dpkg/info/libusb-1.0-0:armhf.list
-rw-r--r-- 1 root root  355 Mar  4 12:57 /var/lib/dpkg/info/libustr-1.0-1:armhf.list
-rw-r--r-- 1 root root  268 Mar  4 12:57 /var/lib/dpkg/info/libuuid1:armhf.list
-rw-r--r-- 1 root root  290 May 20 03:35 /var/lib/dpkg/info/libva1:armhf.list
-rw-r--r-- 1 root root  314 May 20 03:35 /var/lib/dpkg/info/libva-drm1:armhf.list
-rw-r--r-- 1 root root  318 May 20 03:35 /var/lib/dpkg/info/libva-x11-1:armhf.list
-rw-r--r-- 1 root root  522 May 20 03:35 /var/lib/dpkg/info/libvdpau1:armhf.list
-rw-r--r-- 1 root root  404 May 20 03:36 /var/lib/dpkg/info/libvorbis0a:armhf.list
-rw-r--r-- 1 root root  330 May 20 03:36 /var/lib/dpkg/info/libvorbisenc2:armhf.list
-rw-r--r-- 1 root root  335 May 20 03:36 /var/lib/dpkg/info/libvorbisfile3:armhf.list
-rw-r--r-- 1 root root  403 May 20 03:36 /var/lib/dpkg/info/libvpx4:armhf.list
-rw-r--r-- 1 root root  317 May 20 03:36 /var/lib/dpkg/info/libwavpack1:armhf.list
-rw-r--r-- 1 root root  299 May 20 03:36 /var/lib/dpkg/info/libwebp6:armhf.list
-rw-r--r-- 1 root root  317 May 20 03:36 /var/lib/dpkg/info/libwebpmux2:armhf.list
-rw-r--r-- 1 root root  540 May 20 03:34 /var/lib/dpkg/info/libwrap0:armhf.list
-rw-r--r-- 1 root root  11K May 20 03:35 /var/lib/dpkg/info/libx11-data.list
-rw-r--r-- 1 root root  447 May 20 03:36 /var/lib/dpkg/info/libx264-148:armhf.list
-rw-r--r-- 1 root root  223 May 20 03:36 /var/lib/dpkg/info/libx265-95:armhf.list
-rw-r--r-- 1 root root  293 May 20 03:35 /var/lib/dpkg/info/libxau6:armhf.list
-rw-r--r-- 1 root root  293 May 20 03:35 /var/lib/dpkg/info/libxcb1:armhf.list
-rw-r--r-- 1 root root  335 May 20 03:36 /var/lib/dpkg/info/libxcb-render0:armhf.list
-rw-r--r-- 1 root root  317 May 20 03:36 /var/lib/dpkg/info/libxcb-shm0:armhf.list
-rw-r--r-- 1 root root  305 May 20 03:35 /var/lib/dpkg/info/libxdmcp6:armhf.list
-rw-r--r-- 1 root root  540 May 20 03:35 /var/lib/dpkg/info/libxml2:armhf.list
-rw-r--r-- 1 root root  317 May 20 03:36 /var/lib/dpkg/info/libxrender1:armhf.list
-rw-r--r-- 1 root root  321 May 20 03:36 /var/lib/dpkg/info/libxvidcore4:armhf.list
-rw-r--r-- 1 root root  529 May 20 03:36 /var/lib/dpkg/info/libzvbi0:armhf.list
-rw-r--r-- 1 root root 1.2K May 20 03:36 /var/lib/dpkg/info/libzvbi-common.list
-rw-r--r-- 1 root root  454 May 20 03:35 /var/lib/dpkg/info/li`x11-6:armhf.list
-rw-r--r-- 1 root root  20K May 20 00:59 /var/lib/dpkg/info/locales.list
-rw-r--r-- 1 root root 8.5K May 20 00:59 /var/lib/dpkg/info/login.list
-rw-r--r-- 1 root root  304 Mar  4 12:57 /var/lib/dpkg/info/lsb-base.list
-rw-r--r-- 1 root root  676 Mar  4 12:57 /var/lib/dpkg/info/mawk.list
-rw-r--r-- 1 root root 1.9K May 20 03:36 /var/lib/dpkg/info/minidlna.list
-rw-r--r-- 1 root root 1.1K May 20 00:59 /var/lib/dpkg/info/mount.list
-rw-r--r-- 1 root root 5.0K May 20 03:36 /var/lib/dpkg/info/nano.list
-rw-r--r-- 1 root root  739 May 20 00:59 /var/lib/dpkg/info/ncurses-bin.list
-rw-r--r-- 1 root root 5.2K May 20 00:59 /var/lib/dpkg/info/openssl.list
-rw-r--r-- 1 root root  357 Mar  4 12:57 /var/lib/dpkg/info/p7zip.list
-rw-r--r-- 1 root root  336 May 20 00:59 /var/lib/dpkg/info/parted.list
-rw-r--r-- 1 root root  12K May 20 00:59 /var/lib/dpkg/info/passwd.list
-rw-r--r-- 1 root root  37K Mar  4 12:57 /var/lib/dpkg/info/perl-base.list
-rw-r--r-- 1 root root  384 Mar  4 12:57 /var/lib/dpkg/info/pinentry-curses.list
-rw-r--r-- 1 root root 1.7K May 20 00:59 /var/lib/dpkg/info/procps.list
-rw-r--r-- 1 root root 3.9K May 20 03:34 /var/lib/dpkg/info/proftpd-basic.list
-rw-r--r-- 1 root root 3.4K Mar  4 12:57 /var/lib/dpkg/info/psmisc.list
-rw-r--r-- 1 root root  402 May 20 03:37 /var/lib/dpkg/info/qbittorrent-nox.list
-rw-r--r-- 1 root root  370 May 20 03:27 /var/lib/dpkg/info/raspberrypi-bootloader.list
-rw-r--r-- 1 root root 226K May 20 03:27 /var/lib/dpkg/info/raspberrypi-kernel.list
-rw-r--r-- 1 root root 1012 Mar  4 12:57 /var/lib/dpkg/info/raspberrypi-sys-mods.list
-rw-r--r-- 1 root root  231 Mar  4 12:57 /var/lib/dpkg/info/raspbian-archive-keyring.list
-rw-r--r-- 1 root root  531 Mar  4 12:57 /var/lib/dpkg/info/readline-common.list
-rw-r--r-- 1 root root 1.3K May 20 00:59 /var/lib/dpkg/info/resolvconf.list
-rw-r--r-- 1 root root  203 May 20 00:59 /var/lib/dpkg/info/rfkill.list
-rw-r--r-- 2 root root  25M Mar 14  2018 /var/lib/dpkg/info/rsync.list
-rw-r--r-- 1 root root 4.1K Mar  4 12:57 /var/lib/dpkg/info/sed.list
-rw-r--r-- 1 root root  30K May 20 00:59 /var/lib/dpkg/info/systemd.list
-rw-r--r-- 1 root root  482 May 20 00:59 /var/lib/dpkg/info/sysvinit-utils.list
-rw-r--r-- 1 root root  73K May 20 01:00 /var/lib/dpkg/info/tzdata.list
-rw-r--r-- 1 root root  614 May 20 03:34 /var/lib/dpkg/info/ucf.list
-rw-r--r-- 1 root root 4.1K May 20 00:59 /var/lib/dpkg/info/udev.list
-rw-r--r-- 1 root root  592 May 20 00:59 /var/lib/dpkg/info/usbutils.list
-rw-r--r-- 1 root root  11K May 20 00:59 /var/lib/dpkg/info/util-linux.list
-rw-r--r-- 1 root root 4.1K May 20 01:00 /var/lib/dpkg/info/wget.list
-rw-r--r-- 1 root root  373 Mar  4 12:57 /var/lib/dpkg/info/whiptail.list
-rw-r--r-- 1 root root 1.9K May 20 00:59 /var/lib/dpkg/info/wireless-tools.list
-rw-r--r-- 1 root root 2.6K May 20 01:00 /var/lib/dpkg/info/wpasupplicant.list
-rw-r--r-- 1 root root  16K Mar  4 12:57 /var/lib/dpkg/info/xkb-data.list
-rw-r--r-- 1 root root 1.8K Mar  4 12:57 /var/lib/dpkg/info/xz-utils.list
-rw-r--r-- 1 root root  254 Mar  4 12:57 /var/lib/dpkg/info/zlib1g:armhf.list

@MichaIng
Copy link
Owner

@tenryou-17
Okay, so the files for the package that were mentioned in your warning are indeed missing. This should never happen, as now when removing those packages, no files are removed, thus they are orphaned.

Do you have a backup where we could recover the missing list files?

But those warnings did not make the upgrade fail, but:

files list file for package 'dash' is missing final newline

Could be fixed by: echo '' >> /var/lib/dpkg/info/dash.list

And the sources file error could be tried to be solved by cleaning everything and redownloading them:

apt clean
rm -Rf /tmp/apt /var/cache/apt /var/lib/apt/{lists,mirrors,periodic,cdroms.list}
mkdir -p /var/cache/apt/archives
apt update
apt upgrade

What I am thinking about why these files are missing or broken. Perhaps some SDcard corruption. Could you paste: dmesg | tail -20
This allows us to see kernel errors, probably disk I/O errors, if present.

@tenryou-17
Copy link
Author

So no output to paste for the first three lines. Here is the output for apt update:

Get:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease [15.0 kB]
Hit:2 https://archive.raspberrypi.org/debian stretch InRelease
Get:3 http://raspbian.raspberrypi.org/raspbian stretch/main armhf Packages [11.7 MB]
Get:4 http://raspbian.raspberrypi.org/raspbian stretch/contrib armhf Packages [56.9 kB]
Get:5 http://raspbian.raspberrypi.org/raspbian stretch/non-free armhf Packages [95.5 kB]
Get:6 http://raspbian.raspberrypi.org/raspbian stretch/rpi armhf Packages [1,360 B]
Fetched 11.7 MB in 49s (235 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
14 packages can be upgraded. Run 'apt list --upgradable' to see them.

And here is the output for apt upgrade:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  libbluray1
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  libbluray2
The following packages will be upgraded:
  libavcodec57 libavformat57 libavutil55 libdbus-1-3 libexpat1 libraspberrypi-bin libraspberrypi0 libssl1.0.2 libssl1.1 libswresample2 openssl
  proftpd-basic raspberrypi-bootloader raspberrypi-kernel
14 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 51.0 MB of archives.
After this operation, 2,044 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:2 https://archive.raspberrypi.org/debian stretch/main armhf libavutil55 armhf 7:3.2.14-1~deb9u1+rpt1 [239 kB]
Get:6 https://archive.raspberrypi.org/debian stretch/main armhf libswresample2 armhf 7:3.2.14-1~deb9u1+rpt1 [94.6 kB]
Get:1 http://debian.anexia.at/raspbian/raspbian stretch/main armhf libssl1.0.2 armhf 1.0.2s-1~deb9u1 [899 kB]
Get:9 https://archive.raspberrypi.org/debian stretch/main armhf libraspberrypi-bin armhf 1.20190718~stretch-1 [338 kB]
Get:10 https://archive.raspberrypi.org/debian stretch/main armhf raspberrypi-kernel armhf 1.20190718~stretch-1 [35.4 MB]
Get:8 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian stretch/main armhf proftpd-basic armhf 1.3.5b-4+deb9u1 [2,377 kB]
Get:3 http://debian.anexia.at/raspbian/raspbian stretch/main armhf libssl1.1 armhf 1.1.0k-1~deb9u1 [1,117 kB]
Get:4 http://debian.anexia.at/raspbian/raspbian stretch/main armhf libdbus-1-3 armhf 1.10.28-0+deb9u1 [176 kB]
Get:5 http://debian.anexia.at/raspbian/raspbian stretch/main armhf libexpat1 armhf 2.2.0-2+deb9u2 [62.3 kB]
Get:7 http://debian.anexia.at/raspbian/raspbian stretch/main armhf openssl armhf 1.1.0k-1~deb9u1 [712 kB]
Get:11 https://archive.raspberrypi.org/debian stretch/main armhf libraspberrypi0 armhf 1.20190718~stretch-1 [843 kB]
Get:12 https://archive.raspberrypi.org/debian stretch/main armhf raspberrypi-bootloader armhf 1.20190718~stretch-1 [3,579 kB]
Get:13 https://archive.raspberrypi.org/debian stretch/main armhf libavcodec57 armhf 7:3.2.14-1~deb9u1+rpt1 [4,162 kB]
Get:14 https://archive.raspberrypi.org/debian stretch/main armhf libbluray2 armhf 1:1.0.2-1 [128 kB]
Get:15 https://archive.raspberrypi.org/debian stretch/main armhf libavformat57 armhf 7:3.2.14-1~deb9u1+rpt1 [874 kB]
Fetched 51.0 MB in 1min 45s (483 kB/s)
Preconfiguring packages ...
dpkg: warning: files list file for package 'libnpth0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'console-setup' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libcairo2:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libnettle6:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'keyboard-configuration' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libidn2-0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libip4tc0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'raspi-copies-and-fills' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libxext6:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libxfixes3:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'sudo' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libreadline7:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'multiarch-support' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'systemd-sysv' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libsoxr0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'unzip' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libx11-6:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libdns-export162' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'wireless-regdb' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libicu57:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'tar' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libssl1.0.2:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'sensible-utils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libssh2-1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'init-system-helpers' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ncurses-base' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libqt5xml5:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libssl1.1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libbluray1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libkrb5-3:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libtext-charwidth-perl' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libgme0:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ethtool' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libnewt0.52:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'htop' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'console-setup-linux' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'gnupg-agent' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'cron' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libsepol1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libidn11:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libgcc1:armhf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'netbase' missing; assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting:
 files list file for package 'rsync' contains empty filename
E: Sub-process /usr/bin/dpkg returned an error code (2)

And here is the output for dmesg | tail -20:

[   10.431758] usb 1-1.3: new high-speed USB device number 6 using dwc_otg
[   10.562553] usb 1-1.3: New USB device found, idVendor=1058, idProduct=1100, bcdDevice= 1.65
[   10.562560] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   10.562565] usb 1-1.3: Product: My Book
[   10.562569] usb 1-1.3: Manufacturer: Western Digital
[   10.562574] usb 1-1.3: SerialNumber: 57442D574341553436303331313338
[   10.563004] usb-storage 1-1.3:1.0: USB Mass Storage device detected
[   10.563276] scsi host1: usb-storage 1-1.3:1.0
[   10.578213] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xCDE1
[   11.592672] scsi 1:0:0:0: Direct-Access     WD       10EAVS External  1.65 PQ: 0 ANSI: 4
[   11.593690] sd 1:0:0:0: Attached scsi generic sg1 type 0
[   11.593790] sd 1:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
[   11.594522] sd 1:0:0:0: [sdb] Write Protect is off
[   11.594552] sd 1:0:0:0: [sdb] Mode Sense: 21 00 00 00
[   11.595186] sd 1:0:0:0: [sdb] No Caching mode page found
[   11.595204] sd 1:0:0:0: [sdb] Assuming drive cache: write through
[   11.632864]  sdb: sdb1
[   11.636025] sd 1:0:0:0: [sdb] Attached SCSI disk
[   15.953088] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[   16.367055] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)

I don't have a backup, but my setup is fairly simple. It would be trivial to reflash the SD card with the latest DietPi image.

I have found that every couple of months I run into this problem where it won't update. I just end up reflashing the card and think nothing of it, because my setup is really simple. However, now I think it is a pattern which is why I reported it.

@MichaIng
Copy link
Owner

MichaIng commented Aug 11, 2019 •
edited
Loading

@tenryou-17

files list file for package 'rsync' contains empty filename

Can you paste: cat /var/lib/dpkg/info/rsync.list
Lets see how this kind of damage looks like.
EDIT: Ah forget it, it's 25 MiB large, that is multiple times the whole scrollback buffer 😄:

25M Mar 14 2018 /var/lib/dpkg/info/rsync.list

  • That is clearly a file corruption, as these files are only some KiB large. So fix it by removing it.

About update issues. Yeah this is all on APT level currently and there are several possibilities why APT can fail. However in your case something either touched those .list files which should nothing do else of APT/DPkg, or some disk I/O corrupted them. And for disk I/O errors again there are a bunch of possible reasons, including voltage issues (unstable/insufficient power supply) or simply bad quality or old dying SDcard.

However your dmesg (kernel log) looks clean.
So as long as no other more important files are affected, this is not a big issue for now. We just need to go through all reported APT errors and fix or remove the related faulty .list file. Those are re-created as fast as the package is upgraded another time, we could even reinstall them after having all errors fixed, which will then re-created the list files as well.

Okay as above, please do:

rm /var/lib/dpkg/info/rsync.list
apt upgrade # Next try 😉

@tenryou-17
Copy link
Author

I think the

rm /var/lib/dpkg/info/rsync.list
apt upgrade

may have been the straw that broke the camels back. The apt upgrade failed again. I tried to reboot the Pi to try again, but alas it does not boot. Unfortunately, I didn't save the output on my first run. Before I reformat my card and do a fresh install, is there anything I can get off the SD card? A log file or something that may be useful?

Thanks for your help!

@MichaIng
Copy link
Owner

@tenryou-17
Sorry to hear, sadly not unexpected, as SDcards are very prone to file system errors, and it would have been lucky if only those list files would have been affected.

The downside or RAM log is that there are no system logs available after power off. You could check /var/tmp/dietpi/logs for the logs of DietPi boot services. Perhaps they show some error message as well that gives some more detail what exactly failed at boot. Note that some of those logs are a bid ugly due to colour codes, the output is formatted for console print in the first place and just split to log file 😉.

To have better control about file system errors:

  • tune2fs -c 1 /dev/mmcblk0p2 This forces a fsck and in case repair on the root file system on every boot. Boot takes slightly longer, if no issues are found, but it can safe your system before one corrupted file leads to issues elsewhere.
  • When you are working on console, have a look at dmesg from time to time. This is the kernel log and contains file system errors. You will see red line entries with something about I/O error and the affected inode ("physical" file identifier).
  • Reduce writes to the SDcard as much as possible. If you have enough free RAM (>=500M free on idle), you could move APT cache and APT list files (not the corrupted ones, but the larger release files that are downloaded when doing apt update): dietpi-config > Advanced Options > APT cache
    • APT cache has been disabled already with last version, it should just never be on an SDcard, since it leads to 50 - 100 MiB writes on every apt update.
    • APT lists as said, can be moved to RAM as well, and will consume 50 - 100 MiB RAM. statically. So this should only be done if there is really more then enough free, meaning swap file is nearly not in use. Otherwise if the swap file creates the disk writes on the other end, the step would be useless 😉.
    • APT archives are the location where APT/DPkg downloads deb packages, before those are being installed. When you do a large software installs, dunno Java JDK can be large, some header/source/dev packages, then this can theoretically get quite large. When doing apt upgrade/install you get some output saying "Need to get 257 kB of archives." (just now on my RPi), this size is what this archives directory will have. So when moving it to RAM, you should either have plenty of free RAM or check those numbers and available RAM before doing APT upgrades/installs. But it would safe again mentioned amount of disk writes.
    • But more important then above is to have dietpi_userdata on an external drive. Not only since these data might be what not want to loose, but as well because some software titles do heavy writes there, e.g. MariaDB databases are on my system the part that causes VERY most of all disk writes, only by a single Nextcloud instance, used by myself and wife with ~5 clients (including calendars) connected.

And of course you could even move the whole system to a USB drive. Since RPi3 you cannot only move the root fs to some USB drive, but it can boot from there, after programming it (can be done via dietpi-config as well). From RPi3+ on this programming is not even required anymore.
So SDcard images can then be flashed on an USB drive instead and simply booted from there. Many users avoid to use an SDcard at all by this, due to the known limited lifetime and limited speed as well, usually.

@tenryou-17
Copy link
Author

So in my haste to get my system up and running after work, I totally forgot to check /var/tmp/dietpi/logs :P

I have never heard about tune2fs -c 1 /dev/mmcblk0p2 before. I will definitely read up on it. It seems like a good tool to use (always learning :))

Your points about safeguarding against file system errors are excellent. Thank you so much. Going forward I will make it a point to keep checking dmesg from time to time. I do have a USB 3.0 disk connected to my Pi. I will move the FS over to there and monitor if anything happens.

Thanks very much for your support @MichaIng. I appreciate the time you've taken to craft such informative responses. The community appreciates you <3

@MichaIng
Copy link
Owner

@tenryou-17
Jep, tune2fs -c <count> /dev/<name> adjust the amount of regular mounts, before systemd runs an fsck on that file system. By default this is set to 60 AFAIK.
An fsck is otherwise forced, if the kernel reported an error and set a dirty bit on that block device. But that does not always work reliable. I sometimes had file system errors after boot and no fsck has been applied. So setting it to 1 is failsafe, regardless how/when the corruption has occurred, or if e.g. one fsck run did not fix everything.

Important for this to work is that in /etc/fstab, the fsck bit is set, which is the last integer of a mount entry. 0 means that systemd never does any fsck, so the tune2fs count is ignored, even dirty bits, AFAIK. Values larger then 1 mean lower priority, so the drives with 1 will be checked first, 2 afterwards etc. By default on DietPi we set it to 1 for root and boot file system only. External drive checks can take very long, so that should not happen on boot without user knowledge, and those can be checked from the running system without issues.

Important as well to know is that RPi kernel/bootloader has an integrated fsck. It runs before the init system (systemd) has been loaded and can be forced for every boot by adding fsck.mode=force to /boot/cmdline.txt. I once ran into an fsck loop. Somehow systemd made an fsck, did a reboot, then bootloader did an fsck as well, reboot, then systemd again. They don't know each others run and some flag that would prevent the loop for each call, got reset or something like that. So both must not be forced to run on every boot, at best just one should be enabled at all. Theoretically the bootloader call should be safer, since less of the system is loaded, on the other hand the results are not logged very well, compared to the systemd call, where journalctl gives pretty nice details about the check and repair steps, in case done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tenryou-17 @MichaIng