Avoid depth infinity propfind for e2ee #2572

er-vin · 2020-10-20T15:16:51Z

OK, so after all I'll go for changing the depth of the incriminated PROPFIND call and do the recursive traversal by hand (which means more roundtrips which is a bit unfortunate). I finally did this for several reasons:

we get the fix delivered faster in the hand of users;
even if we optimized it on the server to somehow complete, it'd still be potentially expensive;
also I found today this depth was used in other code paths than the first call when the sync starts which made things worse (and that was clearly the client at fault there so needed fixing anyway).

Fixes #2347
Fixes nextcloud/end_to_end_encryption#189

Signed-off-by: Kevin Ottens <[email protected]>

If there's more than one job we need to unite the maps not simply overwrite them. Signed-off-by: Kevin Ottens <[email protected]>

Signed-off-by: Kevin Ottens <[email protected]>

This way we avoid the expensive SQL query on the server at the price of more round-trips since we're doing the recursive traversal by hand now. Also it turns out this depth was used for all the other propfind calls during sync when we want fresher information regarding a folder. This was very inefficient in all cases and won't happen anymore. Signed-off-by: Kevin Ottens <[email protected]>

nextcloud-desktop-bot · 2020-10-21T08:05:54Z

AppImage file: Nextcloud-PR-2572-10cb4170c7354c919a415f1082cf7bb0618ef988-x86_64.AppImage

To test this change/fix you can simply download above AppImage file and test it.

Please make sure to quit your existing Nextcloud app and backup your data.

epikurist · 2020-10-21T09:25:22Z

Thank you guys! Really appreciate the work. I have been waiting for 2,5 years to have the end-to-end encryption.

er-vin · 2020-10-21T11:58:18Z

/backport to stable-3.0

bcutter · 2020-10-23T14:11:37Z

I tested v3.1.0.15934-daily-20201022.

Behavior changed:

After an initial sync after starting desktop client - which put server at 100 % load for about 30 seconds, desktop client stays in state „waiting - preparing to sync“ and will never re-sync (waited 15 minutes).

Switched back to latest v3.0.2 where one single sync progress takes 10 minutes while putting only one core to 100 %, but at least is syncing even it takes veeeery long.

Is this fix part of the daily I tested?

kruthoff · 2020-10-25T18:07:35Z

Same here. I'm testing the AppImage provided above. Sync scans folders and ends in waiting state. It doesn't work as expected.

er-vin · 2020-10-26T12:07:59Z

Well, at least it doesn't drive the server crazy anymore which was the intention of working around that Depth parameter. It's clearly hitting something else now. Will need to go through the log exploration etc. dance again.

When doing so please make sure this issue isn't already reported, and if that's not the case please file a new one.

bcutter · 2020-10-30T17:07:44Z

Well, at least it doesn't drive the server crazy anymore which was the intention of working around that Depth parameter. It's clearly hitting something else now. Will need to go through the log exploration etc. dance again.

When doing so please make sure this issue isn't already reported, and if that's not the case please file a new one.

Ehm - but now it DOESN`T SYNC AT ALL (you realized that?) so I assume that´s not really an improvement.
Now as 3.0.3 with this "fix" is released you can expect more users crying - as least as soon as it´s released for auto-update in the client.

er-vin · 2020-11-02T08:44:31Z

Well, at least it doesn't drive the server crazy anymore which was the intention of working around that Depth parameter. It's clearly hitting something else now. Will need to go through the log exploration etc. dance again.
When doing so please make sure this issue isn't already reported, and if that's not the case please file a new one.

Ehm - but now it DOESN`T SYNC AT ALL (you realized that?) so I assume that´s not really an improvement.

Well, it does sync for some. The part of the code I changed is behavior compatible with the old code but uses a depth 1 instead of depth infinity. I really think this is hitting something else now. There's been past reports of "getting stuck" before and after that fix so to me it's a different issue which needs being tackled as well. As mentioned on #1269 I'm interested in getting debug logs to figure it out but so far I'm not hitting it.

bcutter · 2020-11-15T23:39:04Z

Well, at least it doesn't drive the server crazy anymore which was the intention of working around that Depth parameter. It's clearly hitting something else now. Will need to go through the log exploration etc. dance again.
When doing so please make sure this issue isn't already reported, and if that's not the case please file a new one.

Ehm - but now it DOESN`T SYNC AT ALL (you realized that?) so I assume that´s not really an improvement.

Well, it does sync for some. The part of the code I changed is behavior compatible with the old code but uses a depth 1 instead of depth infinity. I really think this is hitting something else now. There's been past reports of "getting stuck" before and after that fix so to me it's a different issue which needs being tackled as well. As mentioned on #1269 I'm interested in getting debug logs to figure it out but so far I'm not hitting it.

How to provide client logs. Content of ".owncloudsync.log" seems not to be debug log level.

er-vin · 2020-11-16T08:46:54Z

Nope that's not the one, but let me give some more details on #1269 about this

hdlineage · 2020-11-28T00:44:20Z

This is not fixed in 3.0.3 client. Enabling E2EE on the server side basically cripples the syncing process on windows desktop.

bcutter · 2020-11-28T11:33:37Z

This is not fixed in 3.0.3 client. Enabling E2EE on the server side basically cripples the syncing process on windows desktop.

One needs to provide client logs so @er-vin can take a look at it, see reference in his post above.

MegaV0lt · 2020-12-06T08:31:53Z

Not fixed on linux. client still hangs

client linux 3.03

MegaV0lt · 2020-12-06T08:44:32Z

@er-vin
Have made a debuglog. should i remov some sensitive data before upload?

er-vin · 2020-12-07T11:06:17Z

@er-vin
Have made a debuglog. should i remov some sensitive data before upload?

Thanks for your efforts. We now know what's going on I think. Feel free to add the logs just in case it'd add something new. If you go for the file drop you don't have to remove sensitive data if you trust us enough to be responsible enough with it, otherwise please remove sensitive data first.

victordariovera · 2020-12-12T02:38:01Z

Here OSX Client v.3.0.3 (over nextcloud v. 19.0.3) no way to sync with E2E. Android client is ok but not on desktop. If I can ofuscate all the personal info in the client logs (a lot of metadata) I will attach it. (I don't know any tool to do it easy)

er-vin · 2020-12-14T08:34:27Z

Alright, I don't like to do support on PRs, but since the chatter is still going on here. For the adventurous who are on Linux, it'd be useful if you could test the AppImage corresponding to #2700 (it's posted there). I think it should finally put the timeouts to rest.

Word of caution: note this is a massive PR, it might kill kittens, burn your house or... well... you get the idea. Use with caution, it should become 3.2 in March so we still have plenty of time to QA it but having early feedback is always useful.

Note it's not going to solve it for the crowd having issues removing E2EE folders. It's a completely different issue and will require it's on PR.

er-vin added this to the Desktop 3.1 milestone Oct 20, 2020

er-vin requested review from rullzer, sbeyer, camilasan, MorrisJobke and DominiqueFuchs October 20, 2020 15:16

er-vin force-pushed the avoid_depth_infinity_propfind_for_e2ee branch 2 times, most recently from e254daa to 19eccc4 Compare October 20, 2020 15:18

camilasan approved these changes Oct 20, 2020

View reviewed changes

Kevin Ottens added 7 commits October 21, 2020 10:00

Add a way to know which folder was listed for encrypted status

cfb66dd

Signed-off-by: Kevin Ottens <[email protected]>

Start managing a list of GetFolderEncryptStatusJob

9aeb704

Signed-off-by: Kevin Ottens <[email protected]>

Change the way we store the job result

8e3ca71

If there's more than one job we need to unite the maps not simply overwrite them. Signed-off-by: Kevin Ottens <[email protected]>

Mention the folder we listed in the debug logs

cbc40b5

Signed-off-by: Kevin Ottens <[email protected]>

Factor out the logic to start a new e2ee status job

399b974

Signed-off-by: Kevin Ottens <[email protected]>

Make sure jobs don't outlive ClientSideEncryption

b5fdbef

Signed-off-by: Kevin Ottens <[email protected]>

er-vin force-pushed the avoid_depth_infinity_propfind_for_e2ee branch from 19eccc4 to 10cb417 Compare October 21, 2020 08:01

er-vin merged commit 60da0ce into master Oct 21, 2020

er-vin deleted the avoid_depth_infinity_propfind_for_e2ee branch October 21, 2020 08:16

backportbot-nextcloud bot added the backport-request label Oct 21, 2020

backportbot-nextcloud bot mentioned this pull request Oct 21, 2020

[stable-3.0] Avoid depth infinity propfind for e2ee #2576

Merged

backportbot-nextcloud bot removed the backport-request label Oct 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid depth infinity propfind for e2ee #2572

Avoid depth infinity propfind for e2ee #2572

er-vin commented Oct 20, 2020

nextcloud-desktop-bot commented Oct 21, 2020

epikurist commented Oct 21, 2020

er-vin commented Oct 21, 2020

bcutter commented Oct 23, 2020

kruthoff commented Oct 25, 2020

er-vin commented Oct 26, 2020

bcutter commented Oct 30, 2020

er-vin commented Nov 2, 2020

bcutter commented Nov 15, 2020

er-vin commented Nov 16, 2020

hdlineage commented Nov 28, 2020

bcutter commented Nov 28, 2020

MegaV0lt commented Dec 6, 2020

MegaV0lt commented Dec 6, 2020

er-vin commented Dec 7, 2020

victordariovera commented Dec 12, 2020

er-vin commented Dec 14, 2020

Avoid depth infinity propfind for e2ee #2572

Avoid depth infinity propfind for e2ee #2572

Conversation

er-vin commented Oct 20, 2020

nextcloud-desktop-bot commented Oct 21, 2020

epikurist commented Oct 21, 2020

er-vin commented Oct 21, 2020

bcutter commented Oct 23, 2020

kruthoff commented Oct 25, 2020

er-vin commented Oct 26, 2020

bcutter commented Oct 30, 2020

er-vin commented Nov 2, 2020

bcutter commented Nov 15, 2020

er-vin commented Nov 16, 2020

hdlineage commented Nov 28, 2020

bcutter commented Nov 28, 2020

MegaV0lt commented Dec 6, 2020

MegaV0lt commented Dec 6, 2020

er-vin commented Dec 7, 2020

victordariovera commented Dec 12, 2020

er-vin commented Dec 14, 2020