Integration tests executed on a real deployment as part of the CICD - Shares

[dlpzx]

Same as for #1220.

This issue is to track the progress for the Shares modules.
It has its own dedicated issue because of the challenge of validating the shares

[dlpzx]

Required tests for basic coverage

For fresh deployments

For each of the following API calls we need to test authorized and unauthorized scenarios as well as all possible configurations (e.g. autoapproval...)

• Add items to request
• Reject request
• Approve request*
  • Folder sharing validation
  • Bucket sharing validation
  • Table sharing validation
• Create Share request
• Revoke items*
  • Folder sharing validation
  • Bucket sharing validation
  • Table sharing validation
• Delete share
  *if times allows implement team roles as well as consumption roles. If it is not possible implement consumption roles and leave team roles as next steps

For backwards compatibility

For updated Dataset stacks with existing shares validate:

• Folder sharing validation
• Bucket sharing validation
• Table sharing validation

Full coverage [WIP]

For fresh deployments

• Edit share request purpose
• Email notifications
• Auto-approval datasets

For backwards compatibility

For updated Dataset stacks:

• Approve request
  • Folder sharing validation
  • Bucket sharing validation
  • Table sharing validation
• Revoke items
  • Folder sharing validation
  • Bucket sharing validation
  • Table sharing validation

[SofiaSazonova]

Current progress
Happy path

[SofiaSazonova]

Happy path tests:

New Shares:

• Create Share request
• Submit request without Items (not allowed)
• Add items to request
• Submit request without autoapproval (status -- Submitted)
• Change request purpose
• Submit request with autoapproval (status -- Approved)
• Reject request
• Change reject purpose
• Approve request
• Request succeeded
• Item health verification
• Folder sharing validation (cal list objects via s3 accesspoint , for same account check write|modify)
• Bucket sharing validation (cal list objects in s3 bucket , for same account check write|modify)
• Table sharing validation (cal perform athena query: select * from db.table , for same account check write|modify)
• Revoke items
• Folder sharing validation (no more access)
• Bucket sharing validation (no more access)
• Table sharing validation (no more access)
• Delete share
• Check, that resources, involved in shares, can not be deleted from data.all, while share is active.

Persistent Shares:

• Update persistent envs and datasets used for shares (made in fixtures)
• Share verification test
• Check item access test
• Revoke share test
• Check no access left
• Add all items back to share
• Share approved/processed successfully
• Share verification test
• Check item access test (no more access)

[SofiaSazonova]

Share 'breaks' tests:

1. S3 bucket policy is manually changed
   1.a. Permissions are insufficient
   1.b. More permissions are granted
2. IAM role policy is manually changed
   1.a. Permissions are insufficient
   1.b. More permissions are granted

• Share verification job finished
• Items are Unhealthy, the Reason is relevant
• Share reapplied successfully
• Access to folders/bucket/tables is restored
• For same account shares: separate checks for Read/Write/Modify

[SofiaSazonova]

Resource are missing test:

1. Share principal IAM role is deleted
2. S3 bucket/ AccessPoint/Folder/Table/DB is deleted

• Share verification job finished
• Items are Unhealthy, the Reason is relevant
• Unhealthy Items can be revoked
• Share can be deleted

[SofiaSazonova]

User Managed policy test:
Share principal - consumption role without data.all managed policies

• Share can not be created, if required policies are not attached and the flag "attach missing policies" is False
• Share can be created, when the policies are attached
• Share is created and policies are attached, when required policies are not attached and the flag "attach missing policies" is True