WebUI: Serialize upload of files to prevent upload failures

[csware]

Uploading several bigger files fails using the WebUI.

Steps to reproduce

1. Login to Nextcloud in browser (such as Firefox)
2. Select several bigger files locally and drag them into Nextcloud WebUI to upload

Expected behaviour

The files are all uploaded (sequentially) and the upload succeeds.

Actual behaviour

All files are uploaded in parallel. Only a part of the files are uploaded correctly. This is especially an issue with less bandwidth and gets even worse if Slowloris mitigations are enabled.

Server configuration

Operating system: Debian 9.11

Web server: Apache/2.4.25 (Debian)

Database: MariaDB 10.1.41-MariaDB

PHP version: 7.3.13

Nextcloud version: 17.0.2

Updated from an older Nextcloud/ownCloud or fresh install: updated

Where did you install Nextcloud from: Nextcloud.org server package

Signing status:

Signing status

No errors have been found.

Are you using external storage, if yes which one: local

Are you using encryption: no

Client configuration

Browser: Mozilla Firefox 71.0

Operating system: Win 10 x64

Logs

Web server error log

Web server error log

IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-890c1b7aa7c4ffa0a62bc2ab7efe9443-1577023169303 HTTP/2.0" 201 367 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-3b5fb1e494cbb5e924b3ffa7053f0a85-1577023169303 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-bb803fbbee1f027a43c397be4e2de1c5-1577023169304 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-8350caf63503c4ecc78664f68ca604a2-1577023169304 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-eba065a5c899f4d6c34478a7727774b6-1577023169304 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-bea842cc9f4ffda3fd6c6480718ff639-1577023169305 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-83665f71f5a86d31116d5276eb9be313-1577023169305 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:32 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-e94160be7d3242f180bff890005a0485-1577023169305 HTTP/2.0" 201 61 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:32 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-5ab4530899554ffb48920ba7f3760f7e-1577023169305 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:32 +0100] "PUT /remote.php/dav/uploads/USERNAME/web-file-upload-e94160be7d3242f180bff890005a0485-1577023169305/0 HTTP/2.0" 201 4041 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "PUT /remote.php/dav/uploads/USERNAME/web-file-upload-3b5fb1e494cbb5e924b3ffa7053f0a85-1577023169303/0 HTTP/2.0" 201 3942 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
...

Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"Kuve8QPS1lGKODpo76hy","level":4,"time":"2019-12-22T15:09:31+01:00","remoteAddr":"IP","user":"USERNAME","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","message":{"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 8642560 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.","Code":0,"Trace":[{"file":"\/var\/www\/PATH\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[null]},{"file":"\/var\/www\/PATH\/apps\/dav\/lib\/Upload\/UploadFolder.php","line":45,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["62914560",null]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":1096,"function":"createFile","class":"OCA\\DAV\\Upload\\UploadFolder","type":"->","args":["62914560",null]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":525,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->","args":["uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560",null,null]},{"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpPut"],[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:PUT",[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/PATH\/apps\/dav\/lib\/Server.php","line":317,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/PATH\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/PATH\/remote.php","line":163,"args":["\/var\/www\/PATH\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/PATH\/apps\/dav\/lib\/Connector\/Sabre\/File.php","Line":228,"CustomMessage":"--"},"userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko\/20100101 Firefox\/71.0","version":"17.0.2.1"}

Browser log

Browser log

I see the following warnings approx. 29K times:

The humanFileSize library is deprecated! It will be removed in nextcloud 19. 3 main.js:1095:57
The escapeHTML library is deprecated! It will be removed in nextcloud 19. 3 main.js:1095:57

[kesselb]

All files are uploaded in parallel.

Are you sure about that? Please test it with 11 files. 10 should be uploaded in parallel. Upload changed from sequential to concurrent: #13855

I think 10 is a very bad default anyway but I was overruled ;) Feel free to open a pull request and change it to a lower value.

[csware]

An solution could be to make this user configurable.

An even better solution: Don't jnust show one progressbar, but allow to collapse it. Then the user could set a limit and maybe even pause each upload indivitually.

[csware]

I tested with 12 files from which onl two succeeded to upload.

[Pingger]

I can confirm this issue still persists in Nextcloud 20 and also on single file uploads. The larger the file the more likely it fails. With a 1GB File there is a 3 in 4 chance it fails. Sometimes in the beginning sometimes near the end.

My personal workaround is currently using curl to upload to dav.

[szaimen]

Is this Issue still valid in NC21.0.2? If not, please close this issue. Thanks! :)

[csware]

As far as I can see with NC21 files are now uploaded in chunks, however, still several ones at a time.

With my current setup I cannot reproduce this issue ATM.

I suppose the issue comes back when internet bandwidth gets more limited (caused by the Slowloris protection of the webserver).

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.