Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Nextcloud DELETS all the local files and starts the syncronisation all over. #41607

Closed
5 of 8 tasks
Byter3 opened this issue Nov 19, 2023 · 7 comments
Closed
5 of 8 tasks
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 27-feedback bug client: 💻 desktop feature: external storage needs info stale Ticket or PR with no recent activity

Comments

@Byter3
Copy link

Byter3 commented Nov 19, 2023

⚠️ This issue respects the following points: ⚠️

Bug description

Nextcloud: 27.1.3 AIO: v7.6.2
Nextcloud delets all the local files from the clients and starts the syncronisation all over. Which is very not funny when you work with terrabytes of data.
The location of the data is a large SMB share mounted in Nextcloud as an external storage and shared to a group. All the users experience he BUG at the same time. Some of the clients deletes everything localy on the clients and some clients just start to sync again every file which is already existing on the client computers. There is no log corresponding to any SMB error. The SMB backend (a seperate NAS device also not showing any errors in it's logs.)
This is an incredibly serious bug.

Steps to reproduce

I guess it's self explenatory in the bug description, but here we go.
Use nextcloud with SMB external storage and wait for the wipe of your client side.

Expected behavior

Not try to resync everything without any trigger. And most importantly not delete any data on the client side, before trying to resync everything unnecessarily.

Installation method

Official All-in-One appliance

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Updated from a MINOR version (ex. 22.1 to 22.2)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "one-click-instance": true,
        "one-click-instance.user-limit": 100,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "cloud.thefxlovers.com",
        "overwriteprotocol": "https",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "cloud.thefxlovers.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "27.1.3.2",
        "overwrite.cli.url": "https:\/\/cloud.thefxlovers.com\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "updater.release.channel": "stable",
        "updatedirectory": "\/nc-updater",
        "loglevel": "2",
        "app_install_overwrite": [
            "nextcloud-aio",
            "gpxedit"
        ],
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_rotate_size": "10485760",
        "log.condition": {
            "apps": [
                "admin_audit"
            ]
        },
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\Image",
            "2": "OC\\Preview\\MarkDown",
            "3": "OC\\Preview\\MP3",
            "4": "OC\\Preview\\TXT",
            "5": "OC\\Preview\\OpenDocument",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\Krita",
            "0": "OC\\Preview\\Imaginary"
        },
        "enable_previews": true,
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "activity_expire_days": "30",
        "simpleSignUpLink.shown": false,
        "share_folder": "\/Shared",
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
        "htaccess.RewriteBase": "\/",
        "files_external_allow_create_new_local": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "allow_local_remote_servers": true,
        "preview_imaginary_url": "http:\/\/nextcloud-aio-imaginary:9000",
        "memories.exiftool": "\/var\/www\/html\/custom_apps\/memories\/bin-ext\/exiftool-amd64-musl",
        "memories.vod.path": "\/var\/www\/html\/custom_apps\/memories\/bin-ext\/go-vod-amd64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "davstorage.request_timeout": 3600,
        "dbpersistent": false
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - app_api: 1.2.2
  - assistant: 1.0.1
  - calendar: 4.5.3
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - collectives: 2.9.1
  - comments: 1.17.0
  - contacts: 5.4.2
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.11.1
  - event_update_notification: 2.2.0
  - external: 5.2.1
  - externalportal: 1.2.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_antivirus: 5.3.0
  - files_external: 1.19.0
  - files_fulltextsearch: 27.0.1
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - fulltextsearch: 27.0.2
  - fulltextsearch_elasticsearch: 27.0.4
  - gpxpod: 5.0.13
  - groupfolders: 15.3.1
  - imageconverter: 1.3.5
  - integration_openai: 1.1.2
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - mail: 3.4.4
  - maps: 1.1.1
  - memories: 6.0.1
  - news: 24.0.0
  - nextcloud-aio: 0.4.0
  - nextcloud_announcements: 1.16.0
  - notes: 4.8.1
  - notifications: 2.15.0
  - notify_push: 0.6.5
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - passwords: 2023.11.30
  - phonetrack: 0.7.6
  - photos: 2.3.0
  - previewgenerator: 5.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recognize: 5.0.3
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - richdocuments: 8.2.2
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - spreed: 17.1.2
  - support: 1.10.0
  - survey_client: 1.15.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - tasks: 0.15.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_nextcloud_notification: 3.8.0
  - twofactor_totp: 9.0.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0
  - user_ldap: 1.17.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

This maybe usefull:

[core] Error: OCP\Files\NotFoundException: /***/****.docx at <<closure>>

 0. /var/www/html/lib/private/Files/Node/Folder.php line 135
    OC\Files\Node\Root->get("/AG/files/WORK/ ... x")
 1. /var/www/html/apps/files_versions/lib/Storage.php line 595
    OC\Files\Node\Folder->get("/WORK/_Eva-Work ... x")
 2. <<closure>>
    OCA\Files_Versions\Storage::OCA\Files_Versions\{closure}("*** sensitive parameters replaced ***")
 3. /var/www/html/apps/files_versions/lib/Storage.php line 610
    array_filter(["*** sensitive  ... "], ["Closure"])
 4. /var/www/html/apps/files_versions/lib/BackgroundJob/ExpireVersions.php line 69
    OCA\Files_Versions\Storage::expireOlderThanMaxForUser("AG")
 5. /var/www/html/lib/private/User/Manager.php line 632
    OCA\Files_Versions\BackgroundJob\ExpireVersions->OCA\Files_Versions\BackgroundJob\{closure}("*** sensitive parameters replaced ***")
 6. /var/www/html/apps/files_versions/lib/BackgroundJob/ExpireVersions.php line 70
    OC\User\Manager->callForSeenUsers(["Closure"])
 7. /var/www/html/lib/public/BackgroundJob/Job.php line 81
    OCA\Files_Versions\BackgroundJob\ExpireVersions->run(null)
 8. /var/www/html/lib/public/BackgroundJob/TimedJob.php line 103
    OCP\BackgroundJob\Job->start(["OC\\BackgroundJob\\JobList"])
 9. /var/www/html/lib/public/BackgroundJob/TimedJob.php line 93
    OCP\BackgroundJob\TimedJob->start(["OC\\BackgroundJob\\JobList"])
10. /var/www/html/cron.php line 152
    OCP\BackgroundJob\TimedJob->execute(["OC\\BackgroundJob\\JobList"], ["OC\\Log"])

at 2023-11-19T20:56:21+00:00

Additional info

No response

@Byter3 Byter3 added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Nov 19, 2023
@joshtrichards
Copy link
Member

I guess it's self explanatory in the bug description, but here we go.

Hi @Byter3 - Unfortunately it is not. The clients, in large part, decide how to sync things. There could also be other factors since upgrading the Server may have also upgraded apps (like groupfolders).

  • What clients are you using?
  • When did this start? (i.e. immediately after updating the server)
  • What client version(s) are you using?
  • Is it impacting non-shared folders?

That said, I'm sorry to hear this is happening.

@Byter3
Copy link
Author

Byter3 commented Nov 19, 2023

Hey,
I'm using the latest Nextcloud client (3.10.1) on windows 11 (build: 22621.2715)
It's hard to say, it's not constant. This is the second time it happened. It happened earlier this week or last weekend can't remember. But last time, it did not delete everything from the client PC and even now only one of the client pc is affected an other one just started resyncing all the files, which is still pretty bad, given it can take up to a full day, to sync that many data down.
As far as I could quickly check only the SMB folder was impacted, but not sure, because we only realized in the middle of the new unprovoked sync.
Thanks for the quick response! This is very inconvenient, it's seriously impacting our workflow.

@joshtrichards
Copy link
Member

Because you're talking about a shared folder, a single client could be triggering something that cascades to other clients.

I suspect you're also using groupfolders (you have it installed anyhow) so that could be a factor:

nextcloud/groupfolders#2577

There haven't been other similar bug reports in the base Server platform though that I'm aware of for the current release.

You will probably need to do some legwork to go through the Server logs to try to pinpoint the initial trigger period and what immediately preceded it in your logs.

As for the clients, you may want to turn on logging to see what's happening there:

https://docs.nextcloud.com/desktop/3.10/troubleshooting.html

@Byter3
Copy link
Author

Byter3 commented Nov 19, 2023

20231119_170759

I realized that, I can see hundreds of this message inside the more severely impacted client's dialog box.

@joshtrichards
Copy link
Member

Sounds like possibly nextcloud/desktop#6099

@szaimen
Copy link
Contributor

szaimen commented Nov 25, 2023

#39707 might help?

@nextcloud-command
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

@nextcloud-command nextcloud-command added the stale Ticket or PR with no recent activity label Mar 5, 2024
@nextcloud-command nextcloud-command closed this as not planned Won't fix, can't repro, duplicate, stale Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 27-feedback bug client: 💻 desktop feature: external storage needs info stale Ticket or PR with no recent activity
Projects
None yet
Development

No branches or pull requests

4 participants