Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cli to remove versions #9472

Closed
kobergj opened this issue Jun 26, 2024 · 9 comments · Fixed by #9497
Closed

Add cli to remove versions #9472

kobergj opened this issue Jun 26, 2024 · 9 comments · Fixed by #9497
Assignees

Comments

@kobergj
Copy link
Collaborator

kobergj commented Jun 26, 2024

Follow up of #9282

We want a cli command to delete all revisions on a storage provider. This is needed for user who use the new DisableVersioning feature and want to clean up their storage. (aka "delete all old versions of my files")

Acceptance criteria:

  • glob all spaces
  • delete all old revisions of files
  • DO NOT TOUCH current the version of the files, only old revisions!
  • include a --dry-run flag that doesn't actually delete the revisions but only prints what it would delete.

@rhafer @micbar @tbsbdr any more acceptance criteria?

@kobergj kobergj self-assigned this Jun 28, 2024
@kobergj kobergj moved this from Qualification to In progress in Infinite Scale Team Board Jun 28, 2024
@okrikos
Copy link

okrikos commented Jul 1, 2024

Would it be possible to add a flag for deleting all old revisions of one particular file? Is the new CLI command able to detect and purge orphan revisions of deleted files?

@kobergj
Copy link
Collaborator Author

kobergj commented Jul 1, 2024

Would it be possible to add a flag for deleting all old revisions of one particular file?

Not in the scope in of this ticket. This is about removing ALL revisions after disabling versioning. What is the usecase of this scenario?

orphan revisions of deleted files

Those should not exist. When a file is deleted the revision should be deleted as well. If this is not the case it is a bug.

To detect orphaned blobs/nodes or other inconsistencies, one should use the ocis backup consistency command. It currently does not have a delete flag, so deleting would need to be done manually. But such a flag could easily be added if there is a need for it.

@okrikos
Copy link

okrikos commented Jul 1, 2024

The primary motivation for this request is to get some control over the disc usage by spaces. Let’s take as an example one of my installations. Three spaces, each with the limit of 1GB, no personal folders. One space size is 395MB, the second 59MB and the last 41 MB = 485MB in total.
du -sh /home/ocis/.ocis/storage/users/spaces gives 12G.
We need at least the very basic tool for removing old revisions in order not to run out of the disc space. Actually, I don’t want to disable versioning – my users love it.

@kobergj
Copy link
Collaborator Author

kobergj commented Jul 1, 2024

I see.

I think the proper solution would be to make it possible to configure a maximum amount of revisions per file. It could remove the oldest version once it hits this limit. If that is something you would like, please open a Feature Request for it. I think others could also benefit of such a feature.

As for the immediate problem, I could simply add a --id flag to this cli command. It would expect an itemid and remove all revisions of this item. Would this be sufficient for a basic (temporary) solution?

@okrikos
Copy link

okrikos commented Jul 1, 2024

It’s perfect, thank you! What would be an appropriate way to get a file id, preferably via the web interface? Or could you display a file id with the --dry-run flag?
Regarding a comprehensive solution, I think that setting up the max number of revisions doesn’t ensure that disk space is not overused. It was done just fine in https://doc.owncloud.com/webui/next/classic_ui/files/version_control.html and as an admin I really love this quote:

The version app never uses more that 50% of the user’s currently available free space.

@kobergj
Copy link
Collaborator Author

kobergj commented Jul 2, 2024

Unfortunately I cannot print the fileid with this command as I am missing essential information. It can be obtained through inspecting web requests (should be in every PROPFIND) or looking into the logs. Here is an example for one:

d419032c-65b9-4f4e-b1e4-0c69a946181d$44b5a63b-540c-4002-a674-0e9c833bbe49!e8a73d49-2e00-4322-9f34-9d7f178577b2

I am afraid there is no simpler way to acquire this information right now. This would need another feature request.

Regarding a comprehensive solution, I think that setting up the max number of revisions doesn’t ensure that disk space is not overused. It was done just fine in https://doc.owncloud.com/webui/next/classic_ui/files/version_control.html and as an admin I really love this quote:

Please open a feature request for this, my paygrade is not hight enough to decide on such features ;)

One space size is 395MB, the second 59MB and the last 41 MB = 485MB in total.
du -sh /home/ocis/.ocis/storage/users/spaces gives 12G.

This sounds a very large discrepancy. It would be interesting to know if ocis backup consistency shows any inconsistencies for those spaces, or if these 11.5G are really actual versions.

@kobergj
Copy link
Collaborator Author

kobergj commented Jul 2, 2024

Unfortunately I cannot print the fileid with this command as I am missing essential information.

Scratch that. I found a way to print the fileid within the command. Will be visible when used with -v flag. Of course it also works when the fileid is copied from web.

I also added possibility to remove all versions on one space - just because it was easily doable.

@okrikos
Copy link

okrikos commented Jul 2, 2024

That is marvelous! Thank you very much!

It would be interesting to know if ocis backup consistency shows any inconsistencies for those spaces

I am afraid, it does not work for me right away:

# ocis backup consistency /home/ocis/.ocis/
No help topic for 'backup'
# ocis version
Version: 4.0.2
Compiled: 2023-10-06 00:00:00 +0000 UTC

Please open a feature request for this

#9512

@kobergj
Copy link
Collaborator Author

kobergj commented Jul 2, 2024

I am afraid, it does not work for me right away:

The backup consistency command needs ocis v5 at least.

Thanks for the feature request. I'll push it when I get a chance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants