Per-user or per-group object storage

[natefoo]

Many people have asked over the years for the ability to separate dataset storage by user or group. This would not be terribly difficult to implement in the object store, but the stumbling block has been what to do with shared data. However, a solution that would probably be "good enough" would be to simply leave data where it is created, even if the original creator of that data removes their instance of it. More complex scenarios or needs could be addressed later.

[selten]

Personally I'd like to have an option like this rather than the aforementioned pull request. I would not like to give my users the option to opt-in to store their Galaxy data somewhere. I prefer to make them store their data in a S3 datastore as defined by me.

[pjbriggs]

Following discussions in the galaxyproject/admins Gitter channel, I'd also have a use case for per-user/per-group object storage which I think fits this issue.

In our case, we would like to be able to assign physical storage for a user or a group to point to different NFS-mounted directories (rather than S3 datastores, as in @selten's comment above), with the assignment managed by the Galaxy admin rather than by the individual users (which appears to be the case in PR #4840).

Ideally we'd prefer a default central or "core" area to be used by users/groups which didn't have target storage explicitly defined; alternatively a hierarchical model where all users have space on the central storage which is supplemented by the per-user/per-group storage would also work for us.

The most significant issue raised on Gitter was how to handle shared data under this model (also noted by @natefoo when creating this issue), with one suggestion to only allow sharing of objects in central storage. For us the preference would be for the physical storage location to be invisible to the user, but I'd imagine this could be difficult to square with sharing data.

[hexylena]

So is this (partially) possible with #6552 ?

Ignoring the "even if the user removes it" bit which doesn't apply to admin controlled object stores.

I was going to fix galaxyproject/training-material#2024 with a "no" but then I checked and saw this PR in my history.

[selten]

I believe technically it could be done.
If one would map users to a destination based on their username or group and subsequently define a different storage target for this.
However, I'd personally like this option to be available in the XML file for object stores, like only-group=[Group Name] as attribute.

[hexylena]

Well, it's fixed per @natefoo @pjbriggs's posts.

I'd definitely love to see some easy XML, let me know when you've added that @selten ;)