-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 Fixes issues with dy-sidecar removal by director-v2 #3029
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3029 +/- ##
========================================
+ Coverage 78.8% 79.7% +0.9%
========================================
Files 698 698
Lines 29226 29245 +19
Branches 3758 3761 +3
========================================
+ Hits 23040 23324 +284
+ Misses 5361 5079 -282
- Partials 825 842 +17
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please review more carefully error handling. Added some tips below ... can provide also some more concrete solutions later.
services/dynamic-sidecar/src/simcore_service_dynamic_sidecar/api/containers_extension.py
Outdated
Show resolved
Hide resolved
@@ -246,6 +247,10 @@ async def service_push_output_ports( | |||
response = await self._client.post( | |||
url, json=port_keys, timeout=self._save_restore_timeout | |||
) | |||
if response.status_code == status.HTTP_404_NOT_FOUND: | |||
# check if the error of the dict is as expected an raise to skip! | |||
if "node_not_found" in response.text: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please think carefully about error handling. Checking for a string in a text in order to classify an error is very unreliable and difficult to maintain!
We had a long discussion about how e.g. pydantic deals with error handling. Let's start using all those inputs ...
Some tips:
- Transform
NodeNotFound
into http-error class but using structured json-response SEE e.g. http error handler inservices/director-v2/src/simcore_service_director_v2/api/errors/http_error.py
- Create a code system to classify errors (We even extended pydantic mechanism in )
|
||
try: | ||
await logged_gather(*tasks) | ||
except NodeportsDidNotFindNodeError: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also do a bigger effort in defining exceptions (e.g. with some sort of hierarchy or code)
If for every error we define a convoluted name for an exception, the number of combinations will not only explode but will be impossible to handle.
For instance, It would be nice to associate this type of exception with a node error, with a node id (e.g. the warning gets that info from the scheduler_data instead of the error itself ...)
services/director-v2/src/simcore_service_director_v2/modules/dynamic_sidecar/errors.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pair reviewed
Kudos, SonarCloud Quality Gate passed! 0 Bugs No Coverage information |
What do these changes do?
Fixes an issue where
director-v2
was unable to removedynamic-sidecars
. If a node was delete from the database the status of thedynamic-sidecar
could not be saved. This case is now correctly handled.Extra:
Related issue/s
How to test
Checklist