fix(performance): Do not set up filesystem on every call #36589

miaulalala · 2023-02-07T14:56:50Z

Removed old OO_FileChunking logic that produced GC- collectable chunks

Added BG Job that will run a garbage collection for every user

Signed-off-by: Anna Larch [email protected]

Resolves: Basic auth requests always set up the filesystem #31357

Summary

Comparison for OCS requests:
https://blackfire.io/profiles/compare/2c3c3699-137e-49a3-a363-e0950bb278d4...41a33cf0-71ee-42aa-9791-9ef74f99e6c3/graph?settings%5Bdimension%5D=wt&settings%5Bdisplay%5D=landscape&settings%5BtabPane%5D=nodes&selected=&callname=main()&constraintDoc=

TODO

Testing

Checklist

Code is properly formatted
Sign-off message is added to all commits
Tests (unit, integration, api and/or acceptance) are included
~~Screenshots before/after for front-end changes~~
Documentation (manuals or wiki) has been updated or is not required
~~Backports requested where applicable (ex: critical bugfixes)~~

lib/base.php

apps/dav/lib/Connector/Sabre/Directory.php

lib/private/Server.php

apps/files/lib/BackgroundJob/FileChunkCleanupJob.php

+	public function run($argument) {
+		$this->userManager->callForSeenUsers(function (IUser $user): void {
+			$this->logger->debug('Running chunk cleanup job for user '. $user->getUID());
+			$fileCache = new File();


apps/files/lib/BackgroundJob/FileChunkCleanupJob.php

+	/**
+	 * This job cleans up all backups except the latest 3 from the updaters backup directory
+	 */
+	public function run($argument): void {


stale

miaulalala · 2023-02-13T15:09:29Z

/rebase

juliusknorr · 2023-02-17T13:32:27Z

Last commit seems to indeed break subfolder listing of mount points :/ Needs another dive into why Depth > 0 requests for dav fail with shares being moved into those subfolders without that then.

Also remove old Oc_FileChunking logis that produced GC- collectable chunks Signed-off-by: Anna Larch <[email protected]>

Signed-off-by: Julius Härtl <[email protected]>

juliusknorr · 2023-02-20T07:59:15Z

I finally figured out why the failure was happening. When using the nodes api in dav we initialise it with the wrong view fake root /user/files instead of ``. This doesn't cause issues as dav rarely calls nodes api endpoints that then access the view. Pushed a fix in this PR to reuse already passed LazyFolder instances. The full fix is separate in #36774

DaphneMuller · 2023-02-21T13:01:15Z

hello @miaulalala
Thank you so much for your work on this pull request! This ticket seems to have the tag 'pending documentation'.
Is there any chance you could clarify what documentation is missing? Is this for admins or for app developers?

miaulalala · 2023-02-21T14:16:24Z

This PR was reverted @DaphneMuller

miaulalala self-assigned this Feb 7, 2023

miaulalala marked this pull request as draft February 7, 2023 14:57

miaulalala added 2. developing Work in progress performance 🚀 labels Feb 7, 2023

miaulalala requested review from come-nc, icewind1991 and juliusknorr February 7, 2023 15:13

icewind1991 previously requested changes Feb 7, 2023

View reviewed changes

lib/base.php Show resolved Hide resolved

miaulalala force-pushed the enh/perf-remove-icache branch from a0a28a9 to aadd950 Compare February 7, 2023 17:01

come-nc approved these changes Feb 7, 2023

View reviewed changes

apps/dav/lib/Connector/Sabre/Directory.php Show resolved Hide resolved

apps/dav/lib/Connector/Sabre/Directory.php Outdated Show resolved Hide resolved

come-nc reviewed Feb 7, 2023

View reviewed changes

lib/private/Server.php Outdated Show resolved Hide resolved

miaulalala requested review from icewind1991 and ChristophWurst February 7, 2023 18:59

github-advanced-security bot found potential problems Feb 7, 2023

View reviewed changes

apps/files/lib/BackgroundJob/FileChunkCleanupJob.php

/**

* This job cleans up all backups except the latest 3 from the updaters backup directory

*/

public function run($argument): void {

Check notice

Code scanning / Psalm

MissingParamType

Parameter $argument has no provided type

miaulalala marked this pull request as ready for review February 7, 2023 19:35

miaulalala force-pushed the enh/perf-remove-icache branch from ac9d919 to 60f1f7f Compare February 7, 2023 19:48

come-nc approved these changes Feb 8, 2023

View reviewed changes

miaulalala force-pushed the enh/perf-remove-icache branch from a7e914a to a7b96c5 Compare February 8, 2023 13:26

juliusknorr mentioned this pull request Feb 9, 2023

perf(dav): Do not call general setupFS on ever dav auth #36626

Merged

4 tasks

miaulalala added pending documentation This pull request needs an associated documentation update 3. to review Waiting for reviews and removed 2. developing Work in progress labels Feb 9, 2023

miaulalala changed the title ~~Do not set up filesystem on every call~~ fix(performance): Do not set up filesystem on every call Feb 9, 2023

miaulalala force-pushed the enh/perf-remove-icache branch from a90f63e to 0b25e61 Compare February 9, 2023 15:08

juliusknorr approved these changes Feb 9, 2023

View reviewed changes

juliusknorr added this to the Nextcloud 26 milestone Feb 9, 2023

juliusknorr force-pushed the enh/perf-remove-icache branch from 6473d0d to 1a7041e Compare February 9, 2023 21:44

juliusknorr force-pushed the enh/perf-remove-icache branch 2 times, most recently from 756cc5e to 270828b Compare February 10, 2023 10:54

nextcloud-command force-pushed the enh/perf-remove-icache branch from 21d3ea8 to cbdd2b2 Compare February 13, 2023 15:31

juliusknorr force-pushed the enh/perf-remove-icache branch 2 times, most recently from 15b85a1 to 248c224 Compare February 14, 2023 21:02

miaulalala and others added 5 commits February 17, 2023 19:18

Do not set up filesystem on every call

5d4efb4

Also remove old Oc_FileChunking logis that produced GC- collectable chunks Signed-off-by: Anna Larch <[email protected]>

tests: Fix test isolation on object storage

20058eb

Signed-off-by: Julius Härtl <[email protected]>

tests: Remove legacy checksum test

3493099

Signed-off-by: Julius Härtl <[email protected]>

tests: Remove legacy chunking tests

ec35650

Signed-off-by: Julius Härtl <[email protected]>

fix: Use passed LazyFolder when building sabre nodes

3bdf332

Signed-off-by: Julius Härtl <[email protected]>

juliusknorr force-pushed the enh/perf-remove-icache branch from 248c224 to 3bdf332 Compare February 17, 2023 18:21

juliusknorr mentioned this pull request Feb 18, 2023

fix: always use proper path on node api when calling the view #36774

Merged

juliusknorr merged commit bba3a1c into master Feb 20, 2023

juliusknorr deleted the enh/perf-remove-icache branch February 20, 2023 07:59

This was referenced Feb 21, 2023

Regression in filesystem handling breaks Talk/ attachment folder #36787

Closed

Revert "fix(performance): Do not set up filesystem on every call" #36788

Merged

juliusknorr mentioned this pull request Feb 21, 2023

Basic auth requests always set up the filesystem #31357

Open

1 task

skjnldsv mentioned this pull request Feb 23, 2023

26.0.0 beta 5 #36834

Merged

ChristophWurst removed the pending documentation This pull request needs an associated documentation update label Mar 16, 2023

icewind1991 mentioned this pull request Jun 19, 2024

chore: remove chunking-v1 #45973

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(performance): Do not set up filesystem on every call #36589

fix(performance): Do not set up filesystem on every call #36589

miaulalala commented Feb 7, 2023 •

edited by juliusknorr

Loading

miaulalala commented Feb 13, 2023

juliusknorr commented Feb 17, 2023

juliusknorr commented Feb 20, 2023

DaphneMuller commented Feb 21, 2023

miaulalala commented Feb 21, 2023

fix(performance): Do not set up filesystem on every call #36589

fix(performance): Do not set up filesystem on every call #36589

Conversation

miaulalala commented Feb 7, 2023 • edited by juliusknorr Loading

Summary

TODO

Checklist

miaulalala commented Feb 13, 2023

juliusknorr commented Feb 17, 2023

juliusknorr commented Feb 20, 2023

DaphneMuller commented Feb 21, 2023

miaulalala commented Feb 21, 2023

miaulalala commented Feb 7, 2023 •

edited by juliusknorr

Loading