Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize file metadata #3697

Merged
merged 23 commits into from
Mar 6, 2023
Merged

Optimize file metadata #3697

merged 23 commits into from
Mar 6, 2023

Conversation

aduffeck
Copy link
Contributor

@aduffeck aduffeck commented Mar 2, 2023

This PR improves the ini backend for file metadata:

  • Improve performance
  • Optionally use a reva cache for storing the metadata, which helps
    tremendously when using distributed file systems, for example
  • Allow for using different metadata backends for different storages

We also switched the s3ng integration tests to the ini backend so we cover both the xattrs and the ini backend at the same time.

Finally it adds support for redis sentinel caches (owncloud/ocis#5645)

@update-docs
Copy link

update-docs bot commented Mar 2, 2023

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

@aduffeck aduffeck force-pushed the optimize-file-metadata branch 3 times, most recently from b1a9658 to 46a5d3d Compare March 2, 2023 14:57
@aduffeck aduffeck marked this pull request as ready for review March 2, 2023 15:44
@aduffeck aduffeck requested review from a team, labkode, ishank011 and glpatcern as code owners March 2, 2023 15:44
Comment on lines +441 to +442
p.ParentID, _ = p.Xattr(prefixes.ParentidAttr)
p.Name, _ = p.Xattr(prefixes.NameAttr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can we ignore an error here?

Copy link
Contributor

@butonic butonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renaming the xattrs package to metadata and adding the Metadata interface to PathLookup makes sense. I see how you are sneaking in a Metadata backend into the Node. ;-)

Still needs some hoops for the revisions and trash, but this is a step forward. ❤️

@aduffeck aduffeck force-pushed the optimize-file-metadata branch from 5a53efd to c680d50 Compare March 3, 2023 16:06
@butonic butonic merged commit 3de7ec1 into cs3org:edge Mar 6, 2023
@butonic butonic deleted the optimize-file-metadata branch March 6, 2023 10:12
@micbar micbar mentioned this pull request May 3, 2023
89 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants