Backup files corrupt with nodejs@20 and CIFS on a FritzNAS #971

Grothesk242 · 2023-07-16T11:21:20Z

When the backup is being uploaded to my Google Drive the system's load goes up and stays at this load level:

Uploading to Google Drive fails.

Versions:

Adapter version: 2.6.23
JS-Controller version: 5.0.8
Node version: v20.4.0
Operating system: Debian 12 'Bookworm' ARM64

12:31 was a manual backup, 12:45 a sheduled one:

2023-07-16 12:31:41.097  - info: backitup.0 (849884) starting. Version 2.6.23 in /opt/iobroker/node_modules/iobroker.backitup, node: v20.4.0, js-controller: 5.0.8
2023-07-16 12:31:41.149  - debug: backitup.0 (849884) Backitup has recognized a linux system
2023-07-16 12:31:41.175  - info: backitup.0 (849884) [iobroker] backup will be activated at 12:45 every 1 day(s)
2023-07-16 12:31:41.261  - debug: backitup.0 (849884) noserverino option: true
2023-07-16 12:31:41.262  - debug: backitup.0 (849884) cifs-mount command: "sudo mount -t cifs -o username=iobroker,password=****,noserverino,rw,uid=iobroker,gid=iobroker,file_mode=0777,dir_mode=0777,vers=3.1.1 //192.168.178.1/fritz.nas/Hitachi-HTS545012B9SA00-01/iobbackups/chet /opt/iobroker/backups"
2023-07-16 12:31:42.327  - debug: backitup.0 (849884) mount successfully completed
2023-07-16 12:31:44.379  - debug: backitup.0 (849884) detect backup file 1 from NAS: iobroker_2023_07_05-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.380  - debug: backitup.0 (849884) detect backup file 2 from NAS: iobroker_2023_07_06-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.381  - debug: backitup.0 (849884) detect backup file 3 from NAS: iobroker_2023_07_07-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.382  - debug: backitup.0 (849884) detect backup file 4 from NAS: iobroker_2023_07_08-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.382  - debug: backitup.0 (849884) detect backup file 5 from NAS: iobroker_2023_07_09-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.382  - debug: backitup.0 (849884) detect backup file 6 from NAS: iobroker_2023_07_10-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.383  - debug: backitup.0 (849884) detect backup file 7 from NAS: iobroker_2023_07_11-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.383  - debug: backitup.0 (849884) detect backup file 8 from NAS: iobroker_2023_07_12-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.383  - debug: backitup.0 (849884) detect backup file 9 from NAS: iobroker_2023_07_14-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.383  - debug: backitup.0 (849884) detect backup file 10 from NAS: iobroker_2023_07_15-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:44.384  - debug: backitup.0 (849884) detect backup file 11 from NAS: iobroker_2023_07_16-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.904  - debug: backitup.0 (849884) detect backup file 1 from googledrive: iobroker_2023_07_15-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.905  - debug: backitup.0 (849884) detect backup file 2 from googledrive: iobroker_2023_07_11-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.905  - debug: backitup.0 (849884) detect backup file 3 from googledrive: iobroker_2023_07_10-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.906  - debug: backitup.0 (849884) detect backup file 4 from googledrive: iobroker_2023_07_09-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.906  - debug: backitup.0 (849884) detect backup file 5 from googledrive: iobroker_2023_07_08-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.906  - debug: backitup.0 (849884) detect backup file 6 from googledrive: iobroker_2023_07_07-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.906  - debug: backitup.0 (849884) detect backup file 7 from googledrive: iobroker_2023_07_06-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.907  - debug: backitup.0 (849884) detect backup file 8 from googledrive: iobroker_2023_07_05-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.907  - debug: backitup.0 (849884) detect backup file 9 from googledrive: iobroker_2023_07_04-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.907  - debug: backitup.0 (849884) detect backup file 10 from googledrive: iobroker_2023_07_03-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:45.909  - debug: backitup.0 (849884) detect last backup file: iobroker_2023_07_15-03_42_20_chet_backupiobroker.tar.gz
2023-07-16 12:31:51.169  - debug: backitup.0 (849884) mount activ... umount in 2 Seconds!!
2023-07-16 12:31:53.231  - debug: backitup.0 (849884) umount successfully completed
2023-07-16 12:33:32.276  - debug: backitup.0 (849884) The local storage check was completed successfully. On the host "chet" are currently 98315 MB free space available!
2023-07-16 12:33:35.155  - debug: backitup.0 (849884) The local storage check was completed successfully. On the host "chet" are currently 98315 MB free space available!
2023-07-16 12:33:35.213  - debug: backitup.0 (849884) Backup has started ...
2023-07-16 12:33:45.214  - debug: backitup.0 (849884) [iobroker/mount] noserverino option: true
2023-07-16 12:33:45.215  - debug: backitup.0 (849884) [iobroker/mount] cifs-mount command: "sudo mount -t cifs -o username=iobroker,password=****,noserverino,rw,uid=iobroker,gid=iobroker,file_mode=0777,dir_mode=0777,vers=3.1.1 //192.168.178.1/fritz.nas/Hitachi-HTS545012B9SA00-01/iobbackups/chet /opt/iobroker/backups"
2023-07-16 12:33:45.322  - debug: backitup.0 (849884) [iobroker/mount] mount successfully completed
2023-07-16 12:33:45.325  - debug: backitup.0 (849884) [iobroker/mount] done
2023-07-16 12:33:49.814  - debug: backitup.0 (849884) [iobroker/iobroker] host.chet 8069 states saved
2023-07-16 12:33:50.000  - debug: backitup.0 (849884) [iobroker/iobroker] host.chet 9963 objects saved
2023-07-16 12:33:53.933  - debug: backitup.0 (849884) [iobroker/iobroker] Backup created: /opt/iobroker/backups/iobroker_2023_07_16-12_33_45_chet_backupiobroker.tar.gz
2023-07-16 12:33:53.935  - debug: backitup.0 (849884) [iobroker/iobroker] This backup can only be restored with js-controller version up from 4.1
2023-07-16 12:33:55.036  - debug: backitup.0 (849884) [iobroker/iobroker] done
2023-07-16 12:33:55.791  - debug: backitup.0 (849884) [iobroker/historyDB] compress from historyDB started ...
2023-07-16 12:34:00.372  - debug: backitup.0 (849884) [iobroker/historyDB] Backup created: /opt/iobroker/backups/historyDB_2023_07_16-12_33_55_chet_backupiobroker.tar.gz
2023-07-16 12:34:00.376  - debug: backitup.0 (849884) [iobroker/historyDB] done
2023-07-16 12:34:00.544  - debug: backitup.0 (849884) [iobroker/zigbee] found zigbee database: zigbee.0
2023-07-16 12:34:00.547  - debug: backitup.0 (849884) [iobroker/zigbee] done
2023-07-16 12:34:01.179  - debug: backitup.0 (849884) [iobroker/javascripts] Created javascript_tmp directory: "/opt/iobroker/backups/tmpScripts"
2023-07-16 12:34:01.368  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: dwd_uwz_nina_warnungen
2023-07-16 12:34:01.377  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: lautstaerke_bad
2023-07-16 12:34:01.378  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: lautstaerke_kueche
2023-07-16 12:34:01.379  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: paketlieferungen
2023-07-16 12:34:01.379  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: pakettruhe_offen
2023-07-16 12:34:01.380  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: ps4_waker
2023-07-16 12:34:01.381  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: sonnenaufgang
2023-07-16 12:34:01.381  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: sonnenuntergang
2023-07-16 12:34:01.382  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: speedtest
2023-07-16 12:34:01.383  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: telefon_klingelt
2023-07-16 12:34:01.383  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: treppenhaus_licht_ble
2023-07-16 12:34:01.384  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: treppenhaus_licht_hub
2023-07-16 12:34:01.385  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: tuerklingel_laeutet
2023-07-16 12:34:01.385  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: vorhaenge-sonnenschutz
2023-07-16 12:34:01.386  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: wohnung_verlassen
2023-07-16 12:34:01.386  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: wohnungstuer_entriegelt
2023-07-16 12:34:01.387  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: awtrix
2023-07-16 12:34:01.388  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: zuhause_verlassen
2023-07-16 12:34:01.388  - debug: backitup.0 (849884) [iobroker/javascripts] found Script: nuki_ext_wohnungstuer_entriegelt
2023-07-16 12:34:01.525  - debug: backitup.0 (849884) [iobroker/javascripts] Try deleting the Javascript tmp directory: "/opt/iobroker/backups/tmpScripts"
2023-07-16 12:34:01.578  - debug: backitup.0 (849884) [iobroker/javascripts] Javascript tmp directory "/opt/iobroker/backups/tmpScripts" successfully deleted
2023-07-16 12:34:01.578  - debug: backitup.0 (849884) [iobroker/javascripts] Backup created: /opt/iobroker/backups/javascripts_2023_07_16-12_34_01_chet_backupiobroker.tar.gz
2023-07-16 12:34:01.579  - debug: backitup.0 (849884) [iobroker/javascripts] done
2023-07-16 12:34:02.332  - debug: backitup.0 (849884) [iobroker/cifs] used copy path: /fritz.nas/Hitachi-HTS545012B9SA00-01/iobbackups/chet
2023-07-16 12:34:02.338  - debug: backitup.0 (849884) [iobroker/cifs] done
2023-07-16 12:34:02.800  - debug: backitup.0 (849884) [iobroker/googledrive] Google Drive: Copy iobroker_2023_07_16-12_33_45_chet_backupiobroker.tar.gz...
2023-07-16 12:39:14.771  - error: host.chet Invalid request getRepository. "callback"(false) or "from"(true) is null
2023-07-16 12:45:10.110  - debug: backitup.0 (849884) The local storage check was completed successfully. On the host "chet" are currently 98314 MB free space available!

The text was updated successfully, but these errors were encountered:

simatec · 2023-07-16T13:12:54Z

I could not reproduce this behavior in my test environments.
Very strange, but I have no idea... Let's reboot... iob fix ...

Grothesk242 · 2023-07-16T14:02:25Z

Nope, just did a iob fix and a reboot, Google Drive Backup is not working.
Backup on the FritzNAS is working, so it's only the GDrive part.
I could narrow it down to nodejs@20. Backup is running as expected with nodejs@18.

simatec · 2023-07-16T15:33:09Z

Hmmm is the Connection to GDrive OK?
Which Node module for GDrive is installed?

ioBroker.backitup/package.json

Line 44 in 87a1347

"google-auth-library": "^8.8.0",

ioBroker.backitup/package.json

Line 41 in 87a1347

"@googleapis/drive": "^5.1.0",

Grothesk242 · 2023-07-16T16:29:53Z

Connection to GDrive is established with nodejs@18, so basically it works.

echad@chet:~ $ cd /opt/iobroker/ && npm ls -a | grep google
│ ├─┬ @googleapis/[email protected]
│ │ └─┬ [email protected]
│ │   ├── [email protected] deduped
│ ├─┬ [email protected]
│ │ │ ├─┬ [email protected]

simatec · 2023-07-16T18:18:48Z

Would have to open an issue there for Node20

https://github.com/googleapis/google-api-nodejs-client

Grothesk242 · 2023-07-16T18:55:18Z

Did you try with the latest version of said modules?

simatec · 2023-07-16T19:12:50Z

yes but only with node18

Grothesk242 · 2023-07-16T19:49:37Z

I would not know how to describe a fault in the modules in their issue tracker.
And I did not find a smilar issue for the moment.

simatec · 2023-07-16T19:54:01Z

I would generally write that the module does not run under Nodes20 and the processor load goes to 100%

Grothesk242 · 2023-07-16T19:57:57Z

How can I integrate the latest modules in my backitup installation? Never understood... Editing package.json / package-lock.json?

simatec · 2023-07-16T20:00:13Z

In package.json, adjust the version numbers and then run an npm install in /opt/iobroker/nodes_modules/iobroker.backitup.

After that just upload and restart the adapter.

Grothesk242 · 2023-07-16T20:28:57Z

Doesn't work here.

echad@chet:/opt/iobroker $ cd /opt/iobroker/ && npm ls -a | grep google
├─┬ @googleapis/[email protected] extraneous
│ └── [email protected] deduped
├─┬ [email protected] extraneous
├─┬ [email protected] extraneous
├─┬ [email protected] extraneous
│ ├── [email protected] deduped
│ ├── [email protected] deduped
│ ├─┬ @googleapis/[email protected]
│ │ └─┬ [email protected]
│ │   ├── [email protected] deduped
│ ├─┬ [email protected]
│ │ │ ├─┬ [email protected]

simatec · 2023-07-16T20:37:22Z

Please install the current iobroker.backitup Version from Github.
All dependencies updated

simatec · 2023-07-16T20:37:52Z

98b7fb0

Grothesk242 · 2023-07-16T20:50:30Z

Na geht doch.... Keine Ahnung was ich da immer falsch mache.

echad@chet:/opt/iobroker $ cd /opt/iobroker/ && npm ls -a | grep google
│ ├─┬ @googleapis/[email protected]
│ │ └─┬ [email protected]
│ │   ├── [email protected] deduped
│ ├─┬ [email protected]
│ │ │ ├─┬ [email protected]

simatec · 2023-07-16T20:59:36Z

And the Upload with node20 is running?

Grothesk242 · 2023-07-16T21:08:34Z

Yes. But it has nothing to do with the google part but with my Fritz!NAS!
When disabling the NAS and putting the backup files in /opt/iobroker/backups it works.
Seems as if GoogleDrive can't access the mounted NAS file system. Backup to the NAS works, backup to Google Drive works, just not in combination.

simatec · 2023-07-16T21:16:03Z

Hmmm I pull a test system on Node20 and test times

Grothesk242 · 2023-07-16T21:17:14Z

I had these messages in 'dmesg' in the past, was never a problem. Now it is, obviously.

[Sun Jul 16 22:51:19 2023] CIFS: Attempting to mount \\192.168.178.1\fritz.nas
[Sun Jul 16 22:51:59 2023] CIFS: Attempting to mount \\192.168.178.1\fritz.nas
[Sun Jul 16 22:52:09 2023] CIFS: VFS: bogus file nlink value 0
[Sun Jul 16 22:52:17 2023] CIFS: VFS: bogus file nlink value 0
[Sun Jul 16 22:52:18 2023] CIFS: VFS: bogus file nlink value 0

Had tried to get rid of the message in the past, but without success.

Grothesk242 · 2023-07-16T21:28:59Z

The HDD attached to the FritzBox is ext4 formatted. I think this is kind of unusual. Will try with NTFS in some days.

simatec · 2023-07-16T21:43:57Z

On my testsystem is all fine

node 20.4
all Adapter an js-controller up2date

`

backitup.0	2023-07-16 23:40:07.714	debug	[iobroker] exec: done
backitup.0	2023-07-16 23:40:07.563	debug	[iobroker/umount] done
backitup.0	2023-07-16 23:40:07.559	debug	[iobroker/umount] umount successfully completed
backitup.0	2023-07-16 23:40:02.477	debug	[iobroker/umount] mount activ, umount is started ...
backitup.0	2023-07-16 23:40:02.312	debug	[iobroker/historyJSON] done
backitup.0	2023-07-16 23:40:02.311	debug	[iobroker/historyJSON] new history json values created
backitup.0	2023-07-16 23:40:02.144	debug	[iobroker/historyHTML] done
backitup.0	2023-07-16 23:40:02.068	debug	[iobroker/historyHTML] new history html values created
backitup.0	2023-07-16 23:40:01.892	debug	[iobroker/whatsapp] done
backitup.0	2023-07-16 23:40:00.970	debug	[iobroker/clean] done
backitup.0	2023-07-16 23:40:00.969	debug	[iobroker/clean] No older backup files are deleted, because this backup was started manually
backitup.0	2023-07-16 23:40:00.817	debug	[iobroker/googledrive] done
backitup.0	2023-07-16 23:39:58.081	debug	[iobroker/googledrive] Google Drive: Copy grafana_2023_07_16-23_39_38_testsys_backupiobroker.tar.gz...
backitup.0	2023-07-16 23:39:53.616	debug	[iobroker/googledrive] Google Drive: Copy influxDB_2023_07_16-23_39_30_testsys_backupiobroker.tar.gz...
backitup.0	2023-07-16 23:39:45.614	debug	[iobroker/googledrive] Google Drive: Copy iobroker_2023_07_16-23_39_23_testsys_backupiobroker.tar.gz...
backitup.0	2023-07-16 23:39:45.160	debug	[iobroker/cifs] done
backitup.0	2023-07-16 23:39:45.160	debug	[iobroker/cifs] used copy path: /iobroker-test/iobroker
backitup.0	2023-07-16 23:39:43.956	debug	[iobroker/grafana] done
backitup.0	2023-07-16 23:39:43.956	debug	[iobroker/grafana] Grafana tmp directory "/opt/iobroker/backups/grafana_tmp" successfully deleted
backitup.0	2023-07-16 23:39:43.563	debug	[iobroker/grafana] Try deleting the Grafana tmp directory: "/opt/iobroker/backups/grafana_tmp"
backitup.0	2023-07-16 23:39:43.562	debug	[iobroker/grafana] Backup created: /opt/iobroker/backups/grafana_2023_07_16-23_39_38_testsys_backupiobroker.tar.gz
backitup.0	2023-07-16 23:39:38.425	debug	[iobroker/grafana] start Grafana backup compress ...
backitup.0	2023-07-16 23:39:37.618	debug	[iobroker/grafana] found Dashboard: pfsense
backitup.0	2023-07-16 23:39:37.249	debug	[iobroker/grafana] Grafana is available ... Status: 200
backitup.0	2023-07-16 23:39:37.218	debug	[iobroker/grafana] start Grafana request ...
backitup.0	2023-07-16 23:39:37.217	debug	[iobroker/grafana] Created datasource directory
backitup.0	2023-07-16 23:39:37.196	debug	[iobroker/grafana] Created dashboards_manually_restore directory
backitup.0	2023-07-16 23:39:37.176	debug	[iobroker/grafana] Created dashboard directory
backitup.0	2023-07-16 23:39:37.154	debug	[iobroker/grafana] Created grafana_tmp directory: "/opt/iobroker/backups/grafana_tmp"
backitup.0	2023-07-16 23:39:37.094	debug	[iobroker/grafana] Start Grafana Backup ...
backitup.0	2023-07-16 23:39:35.888	debug	[iobroker/influxDB] done
backitup.0	2023-07-16 23:39:35.888	debug	[iobroker/influxDB] InfluxDB-Backup for is finish
backitup.0	2023-07-16 23:39:35.887	debug	[iobroker/influxDB] InfluxDB tmp directory "/opt/iobroker/backups/influxDB_2023_07_16-23_39_30_testsys_backupiobroker" successfully deleted
backitup.0	2023-07-16 23:39:34.836	debug	[iobroker/influxDB] Try deleting the InfluxDB tmp directory: "/opt/iobroker/backups/influxDB_2023_07_16-23_39_30_testsys_backupiobroker"
backitup.0	2023-07-16 23:39:34.835	debug	[iobroker/influxDB] Backup created: /opt/iobroker/backups/influxDB_2023_07_16-23_39_30_testsys_backupiobroker.tar.gz
backitup.0	2023-07-16 23:39:30.865	debug	[iobroker/influxDB] InfluxDB Backup tmp directory created
backitup.0	2023-07-16 23:39:30.808	debug	[iobroker/influxDB] Start InfluxDB Backup ...
backitup.0	2023-07-16 23:39:30.806	debug	[iobroker/influxDB] InfluxDB-Backup started ...
backitup.0	2023-07-16 23:39:30.655	debug	[iobroker/iobroker] done
backitup.0	2023-07-16 23:39:29.596	debug	[iobroker/iobroker] This backup can only be restored with js-controller version up from 4.1
backitup.0	2023-07-16 23:39:29.594	debug	[iobroker/iobroker] Backup created: /opt/iobroker/backups/iobroker_2023_07_16-23_39_23_testsys_backupiobroker.tar.gz
backitup.0	2023-07-16 23:39:27.103	debug	[iobroker/iobroker] host.iobroker-test 6721 objects saved
backitup.0	2023-07-16 23:39:26.571	debug	[iobroker/iobroker] host.iobroker-test 5780 states saved
backitup.0	2023-07-16 23:39:23.803	debug	[iobroker/mount] done
backitup.0	2023-07-16 23:39:23.802	debug	[iobroker/mount] mount successfully completed
backitup.0	2023-07-16 23:39:23.693	debug	[iobroker/mount] nfs-mount command: "sudo mount 10.4.1.218:/iobroker-test/iobroker /opt/iobroker/backups"
backitup.0	2023-07-16 23:39:13.691	debug	Backup has started ...

`

Grothesk242 · 2023-07-16T21:49:15Z

I think it has to do with these
CIFS: VFS: bogus file nlink value 0 messages and the ext4 filesystem of the NAS. Will have to dig deeper.
So no issue with the adapter but my setup here.

Grothesk242 · 2023-07-17T16:32:04Z

Another finding:
Backup with nodejs@20 creates a file

iobroker_2023_07_17-18_18_16_chet_backupiobroker.tar.gz
Datei
16 kB
17.07.2023 18:18 Uhr

whereas under nodejs@18 this backup is created:

iobroker_2023_07_17-18_26_10_chet_backupiobroker.tar.gz
Datei
4,71 MB
17.07.2023 18:26 Uhr

Nothing else has been changed, just the nodejs-Version.

simatec · 2023-07-17T17:40:21Z

Please check the Backup from CLI

Grothesk242 · 2023-07-17T17:48:08Z

I'm currently formatting in NTFS and will try again.

Grothesk242 · 2023-07-17T18:39:07Z

Strange. With nodejs@20 the filesizes are too small.
Nodejs@18:

iobroker_2023_07_17-20_16_45_chet_backupiobroker.tar.gz
Datei
4,71 MB
17.07.2023 20:16 Uhr

Nodejs@20:

iobroker_2023_07_17-20_26_48_chet_backupiobroker.tar.gz
Datei
16 kB
17.07.2023 20:26 Uhr

I have absolutely no clue what might be the issue here. A local backup with nodejs@20 has the correct size:

echad@chet:~ $ ls -lah /opt/iobroker/backups/
total 4.8M
drwxrwxr-x+ 2 iobroker iobroker 4.0K Jul 17 20:36 .
drwxrwxr-x+ 6 iobroker iobroker 4.0K Jul 16 22:34 ..
-rw-rw-r--+ 1 iobroker iobroker 4.8M Jul 17 20:36 2023_07_17-20_36_52_backupiobroker.tar.gz

So it must have something to do with the CIFS on the FritzNAS.

simatec · 2023-07-18T11:11:02Z

I need to build CIFS on my NAS for my test environment. Ne Fritzbox I have unfortunately not for testing not available

Grothesk242 · 2023-07-18T11:42:53Z

I think it's very FritzBox specific. The CIFS Server there is a little bit strange... I don't have a different NAS for cross checks. :-(

Grothesk242 · 2023-07-25T20:41:52Z

Keine Lust auf Englisch... ;-)

Hab gerade nochmal mit Backitup bzw. dem CIFS der FritzBox herumgespielt. Also wenn ich außerhalb von Backitup auf den Mountpunkt schreibe (gleiche mount optionen wie im Backitup-Adapter) hab ich gar kein Problem. Die Dateien haben die richtige Größe. Sobald der Adapter versucht zu schreiben ist die Datei nur ca. 16kb groß. Backitup hat auch nur in Verbindung mit nodejs@20 Probleme, mit 18 läuft es einwandfrei.

Ein 'iobroker backup' auf dem Mountpunkt funktioniert aber auch nicht. Ist also irgendwas außerhalb des Adapters.

echad@chet:~ $ ls -lAh /opt/iobroker/backups/
total 93M
-rwxrwxrwx 1 iobroker iobroker  16K Jul 25 23:10 2023_07_25-23_10_22_backupiobroker.tar.gz

simatec · 2023-08-02T08:17:49Z

Das sieht mir nach einem node20 Thema aus...

Grothesk242 · 2023-08-02T08:40:41Z

Ja, scheint mir auch so. Ich schau mal ob die da bei künftigen Releases was umstellen. Für Sentry muß auch beim Dateisystem angepasst werden.

simatec · 2023-08-14T08:46:39Z

Google hat noch mal einiges an den Paketen gebastelt ... Kannst du mal bitte v2.7.0 testen

Grothesk242 · 2023-08-14T08:51:48Z

Kann ich natürlich (später) machen, das Problem stammt aber irgendwie aus der Ecke CIFS, denn das Problem besteht nur wenn das FritzNAS in /opt/iobroker/backups gemountet ist. Direkt aus dem lokalen Filesystem funktioniert alles. simatec ***@***.***> schrieb am Mo., 14. Aug. 2023, 10:46:

…

Google hat noch mal einiges an den Paketen gebastelt ... Kannst du mal bitte v2.7.0 testen — Reply to this email directly, view it on GitHub <#971 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANVE56JPDWBAPJLGI2SU3U3XVHQXVANCNFSM6AAAAAA2L4EYSY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

Grothesk242 · 2023-08-14T17:03:46Z

Nein, Backup auf das FritzNAS weiterhin nicht erfolgreich. Alleine auf GoogleDrive funktioniert.

simatec · 2023-08-14T17:50:38Z

Gut dann müssen wir tatsächlich mal beim mount tiefer schauen

Grothesk242 · 2023-08-14T19:11:53Z

Ist ja nicht mal ein Problem mit dem Adapter, ich kann auch per 'iob backup' keine korrekten Dateien in das manuell gemountete Filesystem schreiben. Und das ganze auch erst mit nodejs@20, mit 18 funktioniert das noch.
Ist jetzt die Frage, ob es am CIFS vom FritzNAS liegt, an nodejs@20, an den mount-Optionen oder am 'iob backup'.

simatec · 2023-08-14T20:11:11Z

Hast mal zum testen andere Dateisysteme versucht? Was ist mit ext4?

Grothesk242 · 2023-08-14T22:04:12Z

Das läuft bereits seit jeher mit ext4. Hab aber auch mal mit NTFS versucht. Resultat war jeweils das gleiche.

simatec · 2023-09-17T21:51:16Z

Kannst du mal bitte die aktuelle Node Version mal testen? Glaube ab 20.6.x sollte der Fehler behoben sein.
Werde es die Tage auch mal testen.

Grothesk242 · 2023-09-17T22:24:56Z

Hab ich natürlich schon getan. War ja auch meine Hoffnung. Die anderen Probleme mit nodesjs haben sich auch erledigt, bleibt nur noch dieses hier. simatec ***@***.***> schrieb am So., 17. Sept. 2023, 23:51:

…

Kannst du mal bitte die aktuelle Node Version mal testen? Glaube ab 20.6.x sollte der Fehler behoben sein. Werde es die Tage auch mal testen. — Reply to this email directly, view it on GitHub <#971 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANVE56NQ4U7I6I2RX3EUOO3X25WF5ANCNFSM6AAAAAA2L4EYSY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

Grothesk242 · 2023-09-27T20:33:17Z

Ich hab das jetzt auch mit nodejs 18.18.0
Vielleicht ist auch einfach der Datenträger hin.

Grothesk242 · 2023-09-30T21:20:16Z

Hab jetzt noch was herumgespielt. Alle Einstellungen in Backitup unverändert, lediglich die nodejs Version ändert sich.
Erfolgreicher Lauf mit nodejs 18.17.1:

Started iobroker ...
[DEBUG] [mount] - noserverino option: true
[DEBUG] [mount] - cifs-mount command: "sudo mount -t cifs -o username=iobroker,password=****,noserverino,rw,uid=iobroker,gid=iobroker,file_mode=0777,dir_mode=0777,vers=3.1.1 //fritz.nas/FRITZ.NAS/Hitachi-HTS545012B9SA00-01/iobbackups/chet /opt/iobroker/backups"
[DEBUG] [mount] - mount successfully completed
[DEBUG] [mount] - done
[DEBUG] [iobroker] - host.chet 7803 states saved

[DEBUG] [iobroker] - host.chet 9577 objects saved

[DEBUG] [iobroker] - Backup created: /opt/iobroker/backups/iobroker_2023_09_30-23_10_22_chet_backupiobroker.tar.gz

[DEBUG] [iobroker] - This backup can only be restored with js-controller version up from 4.1

[DEBUG] [iobroker] - done
[DEBUG] [historyDB] - compress from historyDB started ...
[DEBUG] [historyDB] - Packed 10MB so far...
[DEBUG] [historyDB] - Packed 23MB so far...
[DEBUG] [historyDB] - Backup created: /opt/iobroker/backups/historyDB_2023_09_30-23_10_33_chet_backupiobroker.tar.gz
[DEBUG] [historyDB] - done
[DEBUG] [zigbee] - found zigbee database: zigbee.0
[DEBUG] [zigbee] - done
[DEBUG] [javascripts] - Created javascript_tmp directory: "/opt/iobroker/backups/tmpScripts"
[DEBUG] [javascripts] - found Script: dwd_uwz_nina_warnungen
[DEBUG] [javascripts] - found Script: lautstaerke_bad
[DEBUG] [javascripts] - found Script: lautstaerke_kueche
[DEBUG] [javascripts] - found Script: paketlieferungen
[DEBUG] [javascripts] - found Script: pakettruhe_offen
[DEBUG] [javascripts] - found Script: ps4_waker
[DEBUG] [javascripts] - found Script: sonnenaufgang
[DEBUG] [javascripts] - found Script: sonnenuntergang
[DEBUG] [javascripts] - found Script: speedtest
[DEBUG] [javascripts] - found Script: telefon_klingelt
[DEBUG] [javascripts] - found Script: treppenhaus_licht_ble
[DEBUG] [javascripts] - found Script: treppenhaus_licht_hub
[DEBUG] [javascripts] - found Script: tuerklingel_laeutet
[DEBUG] [javascripts] - found Script: wohnung_verlassen
[DEBUG] [javascripts] - found Script: zuhause_verlassen
[DEBUG] [javascripts] - found Script: nuki_ext_wohnungstuer_entriegelt
[DEBUG] [javascripts] - found Script: stromzaehler
[DEBUG] [javascripts] - found Script: stromzaehler2
[DEBUG] [javascripts] - found Script: frostwarner
[DEBUG] [javascripts] - Try deleting the Javascript tmp directory: "/opt/iobroker/backups/tmpScripts"
[DEBUG] [javascripts] - Javascript tmp directory "/opt/iobroker/backups/tmpScripts" successfully deleted
[DEBUG] [javascripts] - Backup created: /opt/iobroker/backups/javascripts_2023_09_30-23_10_55_chet_backupiobroker.tar.gz
[DEBUG] [javascripts] - done
[DEBUG] [cifs] - used copy path: /FRITZ.NAS/Hitachi-HTS545012B9SA00-01/iobbackups/chet
[DEBUG] [cifs] - done
[DEBUG] [googledrive] - Google Drive: Copy iobroker_2023_09_30-23_10_22_chet_backupiobroker.tar.gz...
[DEBUG] [googledrive] - Google Drive: Copy historyDB_2023_09_30-23_10_33_chet_backupiobroker.tar.gz...
[DEBUG] [googledrive] - Google Drive: Copy zigbee.0_2023_09_30-23_10_55_chet_backupiobroker.tar.gz...
[DEBUG] [googledrive] - Google Drive: Copy javascripts_2023_09_30-23_10_55_chet_backupiobroker.tar.gz...
[DEBUG] [googledrive] - done
[DEBUG] [clean] - No older backup files are deleted, because this backup was started manually
[DEBUG] [clean] - done
[DEBUG] [whatsapp] - done
[DEBUG] [historyHTML] - new history html values created
[DEBUG] [historyHTML] - done
[DEBUG] [historyJSON] - new history json values created
[DEBUG] [historyJSON] - done
[DEBUG] [umount] - mount activ, umount is started ...
[DEBUG] [umount] - umount successfully completed
[DEBUG] [umount] - done
[EXIT] 0

Mit (ab) nodejs 18.18.0 schaut es dann so aus:

Started iobroker ...
[DEBUG] [mount] - noserverino option: true
[DEBUG] [mount] - cifs-mount command: "sudo mount -t cifs -o username=iobroker,password=****,noserverino,rw,uid=iobroker,gid=iobroker,file_mode=0777,dir_mode=0777,vers=3.1.1 //fritz.nas/FRITZ.NAS/Hitachi-HTS545012B9SA00-01/iobbackups/chet /opt/iobroker/backups"
[DEBUG] [mount] - mount successfully completed
[DEBUG] [mount] - done
[DEBUG] [iobroker] - host.chet 7803 states saved

[DEBUG] [iobroker] - host.chet 9577 objects saved

[DEBUG] [iobroker] - Backup created: /opt/iobroker/backups/iobroker_2023_09_30-23_17_29_chet_backupiobroker.tar.gz

[DEBUG] [iobroker] - This backup can only be restored with js-controller version up from 4.1

[DEBUG] [iobroker] - done
[DEBUG] [historyDB] - compress from historyDB started ...
[DEBUG] [historyDB] - Packed 0MB so far...
[DEBUG] [historyDB] - Backup created: /opt/iobroker/backups/historyDB_2023_09_30-23_17_40_chet_backupiobroker.tar.gz
[DEBUG] [historyDB] - done
[DEBUG] [zigbee] - found zigbee database: zigbee.0
[DEBUG] [zigbee] - done
[DEBUG] [javascripts] - Created javascript_tmp directory: "/opt/iobroker/backups/tmpScripts"
[DEBUG] [javascripts] - found Script: dwd_uwz_nina_warnungen
[DEBUG] [javascripts] - found Script: lautstaerke_bad
[DEBUG] [javascripts] - found Script: lautstaerke_kueche
[DEBUG] [javascripts] - found Script: paketlieferungen
[DEBUG] [javascripts] - found Script: pakettruhe_offen
[DEBUG] [javascripts] - found Script: ps4_waker
[DEBUG] [javascripts] - found Script: sonnenaufgang
[DEBUG] [javascripts] - found Script: sonnenuntergang
[DEBUG] [javascripts] - found Script: speedtest
[DEBUG] [javascripts] - found Script: telefon_klingelt
[DEBUG] [javascripts] - found Script: treppenhaus_licht_ble
[DEBUG] [javascripts] - found Script: treppenhaus_licht_hub
[DEBUG] [javascripts] - found Script: tuerklingel_laeutet
[DEBUG] [javascripts] - found Script: wohnung_verlassen
[DEBUG] [javascripts] - found Script: zuhause_verlassen
[DEBUG] [javascripts] - found Script: nuki_ext_wohnungstuer_entriegelt
[DEBUG] [javascripts] - found Script: stromzaehler
[DEBUG] [javascripts] - found Script: stromzaehler2
[DEBUG] [javascripts] - found Script: frostwarner
[DEBUG] [javascripts] - Try deleting the Javascript tmp directory: "/opt/iobroker/backups/tmpScripts"
[DEBUG] [javascripts] - Javascript tmp directory "/opt/iobroker/backups/tmpScripts" successfully deleted
[DEBUG] [javascripts] - Backup created: /opt/iobroker/backups/javascripts_2023_09_30-23_18_04_chet_backupiobroker.tar.gz
[DEBUG] [javascripts] - done
[DEBUG] [cifs] - used copy path: /FRITZ.NAS/Hitachi-HTS545012B9SA00-01/iobbackups/chet
[DEBUG] [cifs] - done
[DEBUG] [googledrive] - Google Drive: Copy iobroker_2023_09_30-23_17_29_chet_backupiobroker.tar.gz...

Und beim google Drive bleiben die kaputten Dateien hängen, die CPU tickt aus und lastet das System komplett aus.
Jetzt müsste man nur wissen, was sich beim Handling von Dateien zwischen 18.17.1 und 18.18.0 bei nodejs getan hat.

Grothesk242 · 2023-09-30T21:21:44Z

Die Dateioperation hier geht schon in die Hose:

[DEBUG] [historyDB] - compress from historyDB started ...
[DEBUG] [historyDB] - Packed 0MB so far...

simatec · 2023-10-08T15:00:36Z

Der Fehler ist bei Node bekannt und tritt aktuell in v18.18.0 und ab Node v20.3.0 auf.
Für Node 18 soll es mit v18.18.1 gefixt werden.
Node 20 ist aktuell noch offen.

nodejs/node#49911

Grothesk242 · 2023-10-10T20:20:44Z

18.18.1 beseitigt das Problem, im 20er-Zweig (20.8.0) ist es noch vorhanden.

simatec · 2023-10-10T20:23:03Z

Ich denke wir sollten im Forum einen Beitrag dazu schreiben und ihn anpinnen... Die Frage kommt nun mehrmals täglich

Grothesk242 · 2023-10-10T21:28:50Z

Wenn man seine Kisten aktuell hält hat man auch keinen Grund für Fragen ...

simatec · 2023-10-11T08:14:36Z

Naja mit Node20 aktuell schon noch ...

Grothesk242 · 2023-10-31T01:32:17Z

Mit der CIFS Mount-Option cache=loose scheint es zu funktionieren. Kann das vielleicht analog wie noserverino angelegt werden?

Grothesk242 · 2023-11-01T15:52:46Z

Issue resolved with Version 2.9.0

Grothesk242 changed the title ~~Copy to Google Drive creates huge load~~ Copy to Google Drive creates huge load with nodejs@20 Jul 16, 2023

Grothesk242 changed the title ~~Copy to Google Drive creates huge load with nodejs@20~~ Backup files corrupt with nodejs@20 and CIFS on a FritzNAS Sep 12, 2023

Grothesk242 closed this as completed Nov 1, 2023

Backup files corrupt with nodejs@20 and CIFS on a FritzNAS #971

Backup files corrupt with nodejs@20 and CIFS on a FritzNAS #971

Comments

Grothesk242 commented Jul 16, 2023 • edited Loading

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023

Grothesk242 commented Jul 16, 2023 • edited Loading

Grothesk242 commented Jul 16, 2023

simatec commented Jul 16, 2023 • edited Loading

Grothesk242 commented Jul 16, 2023

Grothesk242 commented Jul 17, 2023

simatec commented Jul 17, 2023

Grothesk242 commented Jul 17, 2023

Grothesk242 commented Jul 17, 2023

simatec commented Jul 18, 2023 • edited Loading

Grothesk242 commented Jul 18, 2023

Grothesk242 commented Jul 25, 2023 • edited Loading

simatec commented Aug 2, 2023

Grothesk242 commented Aug 2, 2023

simatec commented Aug 14, 2023

Grothesk242 commented Aug 14, 2023 via email

Grothesk242 commented Aug 14, 2023

simatec commented Aug 14, 2023

Grothesk242 commented Aug 14, 2023 • edited Loading

simatec commented Aug 14, 2023

Grothesk242 commented Aug 14, 2023

simatec commented Sep 17, 2023

Grothesk242 commented Sep 17, 2023 via email

Grothesk242 commented Sep 27, 2023 • edited Loading

Grothesk242 commented Sep 30, 2023

Grothesk242 commented Sep 30, 2023

simatec commented Oct 8, 2023

Grothesk242 commented Oct 10, 2023

simatec commented Oct 10, 2023

Grothesk242 commented Oct 10, 2023

simatec commented Oct 11, 2023

Grothesk242 commented Oct 31, 2023

Grothesk242 commented Nov 1, 2023

Grothesk242 commented Jul 16, 2023 •

edited

Loading

Grothesk242 commented Jul 16, 2023 •

edited

Loading

simatec commented Jul 16, 2023 •

edited

Loading

simatec commented Jul 18, 2023 •

edited

Loading

Grothesk242 commented Jul 25, 2023 •

edited

Loading

Grothesk242 commented Aug 14, 2023 •

edited

Loading

Grothesk242 commented Sep 27, 2023 •

edited

Loading