Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛♻️ Better context clenup #2586

Merged

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented Oct 14, 2021

What do these changes do?

Fixes an issue where the some pending coroutines were not properly cleaned up. Only patched director-v2, since director-v0's servicelib is frozen.

tests/test_async_utils.py::test_context_aware_function_sometimes_fails
  /home/crespo/devp/osparc-simcore/.venv/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <coroutine object run_sequentially_in_context.<locals>.internal.<locals>.wrapper.<locals>.worker at 0x7fe0276caf40>
  
  Traceback (most recent call last):
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/queues.py", line 163, in get
      await getter
  GeneratorExit
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/crespo/devp/osparc-simcore/packages/service-library/src/servicelib/async_utils.py", line 93, in worker
      awaitable = await in_q.get()
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/queues.py", line 165, in get
      getter.cancel()  # Just in case getter is not done yet.
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
      self._check_closed()
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
      raise RuntimeError('Event loop is closed')
  RuntimeError: Event loop is closed
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

tests/test_async_utils.py::test_context_aware_wrong_target_args_name
  /home/crespo/devp/osparc-simcore/.venv/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <coroutine object run_sequentially_in_context.<locals>.internal.<locals>.wrapper.<locals>.worker at 0x7fe0276cae40>
  
  Traceback (most recent call last):
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/queues.py", line 163, in get
      await getter
  GeneratorExit
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/crespo/devp/osparc-simcore/packages/service-library/src/servicelib/async_utils.py", line 93, in worker
      awaitable = await in_q.get()
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/queues.py", line 165, in get
      getter.cancel()  # Just in case getter is not done yet.
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
      self._check_closed()
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
      raise RuntimeError('Event loop is closed')
  RuntimeError: Event loop is closed
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

tests/test_async_utils.py::test_context_aware_measure_serialization
  /home/crespo/devp/osparc-simcore/.venv/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <coroutine object run_sequentially_in_context.<locals>.internal.<locals>.wrapper.<locals>.worker at 0x7fe0274edcc0>
  
  Traceback (most recent call last):
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/queues.py", line 163, in get
      await getter
  GeneratorExit
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/crespo/devp/osparc-simcore/packages/service-library/src/servicelib/async_utils.py", line 93, in worker
      awaitable = await in_q.get()
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/queues.py", line 165, in get
      getter.cancel()  # Just in case getter is not done yet.
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
      self._check_closed()
    File "/home/crespo/.pyenv/versions/3.8.10/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
      raise RuntimeError('Event loop is closed')
  RuntimeError: Event loop is closed
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

Related issue/s

How to test

Checklist

@GitHK GitHK self-assigned this Oct 14, 2021
@GitHK GitHK added the bug buggy, it does not work as expected label Oct 14, 2021
@codecov
Copy link

codecov bot commented Oct 14, 2021

Codecov Report

Merging #2586 (8ee0b6d) into master (aff0b8e) will decrease coverage by 0.0%.
The diff coverage is 88.5%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #2586     +/-   ##
========================================
- Coverage    77.2%   77.1%   -0.1%     
========================================
  Files         619     618      -1     
  Lines       23944   23910     -34     
  Branches     2359    2356      -3     
========================================
- Hits        18486   18452     -34     
+ Misses       4825    4824      -1     
- Partials      633     634      +1     
Flag Coverage Δ
integrationtests 65.8% <100.0%> (-0.2%) ⬇️
unittests 71.7% <88.5%> (+0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ages/service-library/src/servicelib/async_utils.py 94.3% <86.6%> (-5.7%) ⬇️
...ore_service_director_v2/api/routes/computations.py 87.5% <100.0%> (ø)
...-v2/src/simcore_service_director_v2/core/events.py 100.0% <100.0%> (ø)
...webserver/computation_comp_tasks_listening_task.py 83.6% <0.0%> (-2.1%) ⬇️
.../src/simcore_service_webserver/director_v2_core.py 67.9% <0.0%> (-1.6%) ⬇️
...ce_webserver/resource_manager/garbage_collector.py 73.9% <0.0%> (+0.8%) ⬆️

@GitHK GitHK added a:director-v2 issue related with the director-v2 service a:services-library issues on packages/service-libs labels Oct 14, 2021
@GitHK GitHK requested a review from sanderegg October 18, 2021 11:37
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice addition! just one minor thing for readability.

@GitHK GitHK requested a review from sanderegg October 18, 2021 13:33
@GitHK GitHK requested a review from mguidon October 19, 2021 06:37
@GitHK GitHK merged commit c215bf9 into ITISFoundation:master Oct 20, 2021
@GitHK GitHK deleted the fix-cleanup-run_sequentially_in_context branch October 20, 2021 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:director-v2 issue related with the director-v2 service a:services-library issues on packages/service-libs bug buggy, it does not work as expected
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants