Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NC upgrade erased entire data folder content #2132

Closed
MarcS1975 opened this issue Dec 28, 2023 · 11 comments · Fixed by #2133
Closed

NC upgrade erased entire data folder content #2132

MarcS1975 opened this issue Dec 28, 2023 · 11 comments · Fixed by #2133

Comments

@MarcS1975
Copy link

I have recently tried to upgrade from 27.0.1.2 to 28.0.1.1 and this action erased all content of my data folder (which was mapped as a docker volume). Luckily I had a backup but I am curious why the NC upgrade procedure did not retain my old data.
You can see from my compose file below, that I have mapped an existing folder 'MEDIA' into the NC container. This MEDIA folder held 1TB of data has been filled prior to installing NC.

- /srv/mypath/mydata/MEDIA:/var/www/html/media

I add MEDIA as External Folder in Nextcloud with file type "local" and access to all data worked fine.
When I decided to upgrade NC, I added :latest behind the NC container name and re-deployed the stack (in Portainer) with the option "pull containers".

Then the NC log files started filing up with the following recurring errors:

28/12/2023
02:06:09
Initializing nextcloud 28.0.1.1 ...
28/12/2023
02:06:09
Upgrading nextcloud from 27.0.1.2 ...
28/12/2023
02:06:10
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
28/12/2023
02:06:10
You may use your browser or the occ upgrade command to do the upgrade
28/12/2023
02:06:10
rsync: [generator] delete_file: rmdir(media) failed: Device or resource busy (16)
28/12/2023
02:06:32
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]

Any ideas why the upgrade resulted in all my data erased (but not the folder itself). Is NC not supposed to keep all persistent data untouched during an upgrade?

My docker compose file is as follows

version: '2'

volumes:
  nextcloud:
  db:

services:
  db:
    image: mariadb:10.6
    restart: always
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
    volumes:
      - /srv/mypath/NC_db:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=xxx
      - MYSQL_PASSWORD=yyy
      - MYSQL_DATABASE=xxx
      - MYSQL_USER=yyy
      - REDIS_HOST=redis
      - REDIS_HOST_PASSWORD=xxx

  app:
    image: nextcloud**:latest**
    restart: always
    ports:
      - 8585:80

    links:
      - db
    volumes:
      - /srv/mypath/NC_app:/var/www/html
      **- /srv/mypath/mydata/MEDIA:/var/www/html/media**
      
    environment:
      - MYSQL_PASSWORD=xxx
      - MYSQL_DATABASE=yyy
      - MYSQL_USER=xxx
      - MYSQL_HOST=db
      - PHP_MEMORY_LIMIT=1024M
      - PHP_UPLOAD_LIMIT=7G

  cron:
    image: rcdailey/nextcloud-cronjob
    restart: always
    network_mode: none
    depends_on:
    - app
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
    - /etc/localtime:/etc/localtime:ro
    environment:
    - NEXTCLOUD_CONTAINER_NAME=nc40-app-1
    - NEXTCLOUD_PROJECT_NAME=
    - NEXTCLOUD_CRON_MINUTE_INTERVAL=5
@martadinata666
Copy link

The path you mount is not in upgrade.exclude. https://github.com/nextcloud/docker/blob/master/upgrade.exclude

@MarcS1975
Copy link
Author

MarcS1975 commented Dec 28, 2023 via email

@martadinata666
Copy link

In simple terms yes. /var/www/html/data/media is your best bet or use nextcloud external storage plugin

@MarcS1975
Copy link
Author

MarcS1975 commented Dec 28, 2023 via email

@martadinata666
Copy link

tldr; yes, it works.

This docker image is not complete packages that needed to run out of the box missing, I dunno why it doesn't want to add samba support and expected users to build one ourself.

https://github.com/nextcloud/docker/tree/master/.examples/dockerfiles/smb

@joshtrichards
Copy link
Member

See https://github.com/nextcloud/docker#persistent-data - last couple paragraphs explains the behavior you encountered.

joshtrichards added a commit that referenced this issue Dec 28, 2023
@joshtrichards
enh(README) Persistent volume upgrade.exclude clarification
fd6f811 
- Make the warning more prominent
- Clarify the language
- Clarity trade-offs

Fixes: #2132  

Signed-off-by: Josh Richards <[email protected]>
@joshtrichards joshtrichards mentioned this issue Dec 28, 2023
Merged
@joshtrichards
Copy link
Member

Check #2133 for a proposed adjustment to the README to make this more prominent and to clarify the existing language. Feedback welcome.

@MarcS1975
Copy link
Author

MarcS1975 commented Dec 28, 2023 via email

@MarcS1975
Copy link
Author

MarcS1975 commented Dec 28, 2023 via email

@joshtrichards
Copy link
Member

thanks - I have done the smbclient addition but I still constantly get
errors in the log files:

[PHP] Error: Undefined array key "attributes" at
/var/www/html/apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php#155
from ? by -- at 28 Dec 2023, 14:00:37
[files] Error: Malformed state response from server
from ? by -- at 28 Dec 2023, 13:54:22

That is either an upstream bug (which I'm fairly certain there is an existing issue open in the server repo for that) or a compatibly issue with some combination of the server-side/OS SMB related components and your SMB server (could even be both). It may or may not be a bug in Nextcloud - there are many SMB protocol/library version combinations.

@MarcS1975
Copy link
Author

MarcS1975 commented Jan 12, 2024 via email

@creekorful creekorful mentioned this issue Oct 2, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

enh(README) Persistent volume upgrade.exclude clarification nextcloud/docker
3 participants
@joshtrichards @martadinata666 @MarcS1975