[Feature]: Adding bucket policy and sub-user support to the operator

[hoptical]

Feature Proposal

Objective

At the moment, the operator is only able to give admin and read-only access with two separate credentials to the users. However, in order to remove the need for Minio users, the operator must be able to create specific credentials with customizable access levels for different users.

Proposed CRDs

S3UserClaim

apiVersion: s3.snappcloud.io/v1alpha1
kind: S3UserClaim
metadata:
  name: s3userclaim-sample
  namespace: s3-operator-test
spec:
  s3UserClass: ceph-default
  readonlySecret: s3-sample-readonly-secret
  adminSecret: s3-sample-admin-secret
  quota:
    maxSize: 1000
    maxObjects: 1000
    maxBuckets: 5
  subusers:
    - myUser1
    - myUser2

This results in creating different credentials for every subuser. The credentials key type can be s3 instead of swift.

S3User

It's the same since s3User instances are created only for every s3UserClaim.

S3Bucket

apiVersion: s3.snappcloud.io/v1alpha1
kind: S3Bucket
metadata:
  name: s3bucket-sample
spec:
  # Only a S3User can be the owner. i.e. s3SubUser can't be.
  s3UserRef: s3userclaim-sample
  s3DeletionPolicy: delete
  s3SubUserBinding:
    - name: myUser1
      access: read
    - name: myUser2
      access: write

Access levers on the subusers:

• read
• write

Reconciliation Flow

When the S3User adds a new subuser, the operator would create a new subuser with access=SubuserAccessNone in S3SubUserSpec which leads to creating credentials (access and secret key) for the new subuser.

When the S3User adds the new Sub3User to the s3Bucket, a bucketPolicy would be added with the defined access level to the bucket for the s3SubUser.

[hoptical]

I decided not to implement the full access since it's meaningless to give the subUser accesses which are not created with s3UserClaim.