✨Comp backend: allow to change to s3 links on on demand clusters (⚠️ devops)

[sanderegg]

What do these changes do?

This PR brings some new ENVs so that on-demand clusters which are fully controlled by us may use S3 links for computations, thus allowing more than 5Gb uploads.

Related issue/s

How to test

DevOps Checklist

Ensure the following ENV are now exposed (and need to be added to the deploys):

COMPUTATIONAL_BACKEND_DEFAULT_CLUSTER_FILE_LINK_TYPE=S3
COMPUTATIONAL_BACKEND_DEFAULT_FILE_LINK_TYPE=PRESIGNED

Introduces a new ENV
COMPUTATIONAL_BACKEND_ON_DEMAND_CLUSTERS_FILE_LINK_TYPE=S3
which can be set with either S3 or PRESIGNED
this will allow to upload more than 5GB when a computational service runs in on-demand clusters

[codecov]

Codecov Report

Merging #4740 (2282708) into master (300dae0) will decrease coverage by 0.1%.
The diff coverage is 80.0%.

@@           Coverage Diff            @@
##           master   #4740     +/-   ##
========================================
- Coverage    87.0%   87.0%   -0.1%     
========================================
  Files        1127    1127             
  Lines       46934   46938      +4     
  Branches     1013    1013             
========================================
+ Hits        40850   40851      +1     
- Misses       5857    5860      +3     
  Partials      227     227             

Flag	Coverage Δ
integrationtests	65.2% <75.0%> (+<0.1%)	⬆️
unittests	84.7% <80.0%> (-0.1%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
.../models_library/api_schemas_directorv2/clusters.py	87.3% <ø> (ø)
...ore_service_director_v2/modules/clusters_keeper.py	50.0% <ø> (ø)
...e_service_director_v2/modules/dask_clients_pool.py	93.4% <66.6%> (-1.2%)	⬇️
...ages/models-library/src/models_library/clusters.py	100.0% <100.0%> (ø)
...2/src/simcore_service_director_v2/core/settings.py	97.5% <100.0%> (+<0.1%)	⬆️

... and 3 files with indirect coverage changes

[codeclimate]

Code Climate has analyzed commit 2282708 and detected 0 issues on this pull request.

View more on Code Climate.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication