🐛♻️ Is638/fixes docker-compose operations bursts and create containers call in dy-sidecar

[pcrespov]

What do these changes do?

• 🐛 FIX: for POST /containers we eliminated docker-compose rm before up. The following was observed
  • dv2 request up
  • sdc runs in background first rm followed by up
  • dv2 re-requests up (not sure why? I guess it since sdc is taking too long ...)
  • sdc runs rm (which probably removes those being created) then restart from stratch up
  • we enter on a cycle ...
  • notice that rm is a close operations and typically takes a bit longer
• ♻️ New pytest_simcore/simcore_service_library_fixtures.py for tests with co-routines decorated with @run_sequentially_in_context
  • Used the same for all services and packages
• ♻️ cleanup docker_compose_utils and settings
• 🐛 FIX: All docker-compose operations in docker_compose_utils API run one at a time @run_sequentially_in_context E.g. avoids running in parallel docker-compose up and config on the same service. I.e. implements

we need to have the fix in the dy-sidecar that prevents running XX docker-compose CLI calls in parallel (currently all these run in parallel and collapses the host RAM/CPUs)

• 🐛 FIXES flaky tests: references with dumps from sets were different

Related issue/s

• improving dynamic-sidecar design osparc-issues#638

How to test

pytest services/dynamic-sidecar/tests/unit/test_core_docker_compose_utils.py::test_burst_calls_to_docker_compose_config

Checklist

[codecov]

Codecov Report

Merging #3187 (a5fba4d) into master (09f85c1) will increase coverage by 1.1%.
The diff coverage is 91.0%.

❗ Current head a5fba4d differs from pull request most recent head 0b1ba85. Consider uploading reports for the commit 0b1ba85 to get more accurate results

@@           Coverage Diff            @@
##           master   #3187     +/-   ##
========================================
+ Coverage    80.9%   82.0%   +1.1%     
========================================
  Files         686     736     +50     
  Lines       30152   31528   +1376     
  Branches     3940    4079    +139     
========================================
+ Hits        24400   25864   +1464     
+ Misses       4920    4829     -91     
- Partials      832     835      +3     

Flag	Coverage Δ
integrationtests	66.4% <100.0%> (-0.1%)	⬇️
unittests	78.5% <89.7%> (+<0.1%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...es/dynamic_sidecar/docker_service_specs/sidecar.py	80.8% <ø> (+6.3%)	⬆️
...ector_v2/modules/dynamic_sidecar/scheduler/task.py	80.2% <ø> (+9.2%)	⬆️
...sidecar/src/simcore_service_dynamic_sidecar/cli.py	0.0% <0.0%> (ø)
...mcore_service_dynamic_sidecar/modules/nodeports.py	25.0% <0.0%> (ø)
...mcore_service_dynamic_sidecar/core/remote_debug.py	33.3% <50.0%> (ø)
...c/simcore_service_dynamic_sidecar/core/settings.py	95.9% <66.6%> (ø)
...rary/src/servicelib/fastapi/requests_decorators.py	80.3% <80.0%> (+1.5%)	⬆️
...ages/service-library/src/servicelib/async_utils.py	88.6% <100.0%> (-3.5%)	⬇️
...service_director_v2/api/routes/dynamic_services.py	87.2% <100.0%> (+5.8%)	⬆️
...-v2/src/simcore_service_director_v2/core/events.py	100.0% <100.0%> (ø)
... and 114 more

[GitHK]

I'm fine if this will improve the situation for now.
Please think about my note below.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication