♻️ Refactor DiskUsage functionality to support efs-guardian

[GitHK]

What do these changes do?

It is now possible to overwrite DiskUsage for some paths mounted by the dynamic-sidecar. This is done via an RPC endpoint and should be used by the efs-guardian service.

Example data to send:

from servicelib.rabbitmq.rpc_interfaces.dynamic_sidecar.disk_usage import update_disk_usage

usage = {
    ".data_assets": DiskUsage.from_efs_guardian(used=10, total=100),
    "home_user_workspace": DiskUsage.from_efs_guardian(used=10, total=100)
}
await disk_usage.update_disk_usage(rpc_client, usage=usage)

Before this change disk usage was propagated to the frontend via dictionary containing paths and quotas. Paths are no longer supported and have been replaced with labels.

The disk_usage object supports the following keys HOST and STATES_VOLUMES which identify:

• the quotas for the disks mounted to the HOST machine where the service was started
• the quotas for the STATES_VOLUMES which can be provided via EFS and are optional.If this entry is not present all quotas are aggregated in the HOST

Currently the frontend only uses the HOST label, which should always be present

Example 1: all volumes exist on same machine

{
  "HOST": {
    "used": 0,
    "free": 1294390525952,
    "total": 1294390525952,
    "used_percent": 0.0
  }
}

Example 2: state volumes are mounted via EFS

{
  "HOST": {
    "used": 0,
    "free": 1294390525952,
    "total": 1294390525952,
    "used_percent": 0.0
  },
  "STATES_VOLUMES": {
    "used": 0,
    "free": 1000000000,
    "total": 1000000000,
    "used_percent": 0.0
  }
}

Related issue/s

• required by 🎨 notify frontend about current efs disk space #6520

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[codecov]

Codecov Report

Attention: Patch coverage is 92.48826% with 16 lines in your changes missing coverage. Please review.

Project coverage is 85.9%. Comparing base (cafbf96) to head (6c52bae).
Report is 647 commits behind head on master.

Files with missing lines	Patch %	Lines
...bitmq/rpc_interfaces/dynamic_sidecar/disk_usage.py	0.0%	12 Missing ⚠️
...amic_sidecar/modules/system_monitor/_disk_usage.py	95.3%	3 Missing ⚠️
...s_library/api_schemas_dynamic_sidecar/telemetry.py	97.8%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #6536      +/-   ##
=========================================
+ Coverage    84.5%   85.9%    +1.3%     
=========================================
  Files          10    1395    +1385     
  Lines         214   59940   +59726     
  Branches       25    1571    +1546     
=========================================
+ Hits          181   51501   +51320     
- Misses         23    8202    +8179     
- Partials       10     237     +227     

Flag	Coverage Δ
unittests	85.9% <92.4%> (+1.3%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ls_library/api_schemas_dynamic_sidecar/__init__.py	100.0% <100.0%> (ø)
...mcore_service_dynamic_sidecar/api/rest/__init__.py	100.0% <100.0%> (ø)
..._service_dynamic_sidecar/api/rest/_dependencies.py	100.0% <100.0%> (ø)
...mcore_service_dynamic_sidecar/api/rest/_routing.py	100.0% <100.0%> (ø)
...ore_service_dynamic_sidecar/api/rest/containers.py	98.3% <100.0%> (ø)
...e_dynamic_sidecar/api/rest/containers_extension.py	94.5% <100.0%> (ø)
..._sidecar/api/rest/containers_long_running_tasks.py	100.0% <100.0%> (ø)
...c/simcore_service_dynamic_sidecar/api/rest/disk.py	100.0% <100.0%> (ø)
...simcore_service_dynamic_sidecar/api/rest/health.py	100.0% <100.0%> (ø)
...ice_dynamic_sidecar/api/rest/prometheus_metrics.py	100.0% <100.0%> (ø)
... and 11 more

... and 1359 files with indirect coverage changes

[odeimaiz]

lgtm

[sanderegg]

I do not quite follow the need to remove the path from the point of view of the sidecar. these are still mounted paths for it so why change that?
maybe we can discuss tomorrow..

[pcrespov]

Thx for the PR. I left some suggestions but I am not sure i fully grasp the idea here. Perhaps you can show me offline? thx

[matusdrobuliak66]

Thanks 👍

[jsaq007]

👍🏼

[sanderegg]

thanks

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud