enable a 300sec in memory cache for resource infos when listing shares on the ocs API #4438
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes. |
|
💥 Acceptance test Core-API-Tests-ocis-storage-6 failed. Further test are cancelled... |
…s on the ocs API Signed-off-by: Jörn Friedrich Dreyer <[email protected]>
Signed-off-by: Jörn Friedrich Dreyer <[email protected]>
Signed-off-by: Jörn Friedrich Dreyer <[email protected]>
butonic
force-pushed
the
ocs-resource-cache-config
branch
from
82b161a to
29d58b3
Compare
|
Kudos, SonarCloud Quality Gate passed!
|
|
Thanks, sounds cool! I would love if your nice explanation in this PR could go to any dev doc to be preserved for the future - it is worth ;-) Any idea where to put it? |
aduffeck
approved these changes
Sep 12, 2022
ownclouders
pushed a commit
that referenced
this pull request
Sep 12, 2022
Author: Jörn Friedrich Dreyer <[email protected]> Date: Mon Sep 12 14:05:24 2022 +0000 enable a 300sec in memory cache for resource infos when listing shares on the ocs API (#4438) * enable a 300sec in memory cache for resource infos when listing shares on the ocs API Signed-off-by: Jörn Friedrich Dreyer <[email protected]> * lint Signed-off-by: Jörn Friedrich Dreyer <[email protected]> * disable ocs cache by default Signed-off-by: Jörn Friedrich Dreyer <[email protected]> Signed-off-by: Jörn Friedrich Dreyer <[email protected]>
ownclouders
pushed a commit
that referenced
this pull request
Sep 13, 2022
Author: Jörn Friedrich Dreyer <[email protected]> Date: Mon Sep 12 14:05:24 2022 +0000 enable a 300sec in memory cache for resource infos when listing shares on the ocs API (#4438) * enable a 300sec in memory cache for resource infos when listing shares on the ocs API Signed-off-by: Jörn Friedrich Dreyer <[email protected]> * lint Signed-off-by: Jörn Friedrich Dreyer <[email protected]> * disable ocs cache by default Signed-off-by: Jörn Friedrich Dreyer <[email protected]> Signed-off-by: Jörn Friedrich Dreyer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Listing the received shares for a user that has 150 incoming shares currently takes 6-10sec. (that is with the gateway cache enabled by locally reverting cs3org/reva#3167)
While the internal listing of shares is in the 140ms ballpark with cs3org/reva#3148 the ocs api also returns file and path information, requiring a stat and a getpath for every share, adding up to hundreds of internal stat requests.
reva can cache the stat requests with an in memory cache or in redis, dropping the response time for the same user from 8sec to 700ms on my machine. The difference between 140ms and 700ms is spent on GetPath calls that we could get rid of completely by requesting the path in the stat request in the fieldmask. something for another PR.
This PR sets the timeout to 300sec / 5min. I don't know if that is a good initial duration. cbox even has a cache warmup that prefetches the list of shared resources, stats them and caches them for the TTL ... which only really makes sense when the TTL is quite large ... anyway, lets start with this.
I absolutely assume the tests will cry over this ... It did, so I disabled the cache by setting TTL to 0 ... The cache can be enabled by setting
FRONTEND_OCS_RESOURCE_INFO_CACHE_TTL=60or more ... but this breaks our tests. In any case as is the PR at least allows enabling and configuring the cache.Related: cs3org/reva#2976