Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure ConnectionPool closes even if network stack swallows cancellation #8928

Merged
merged 1 commit into from
Nov 8, 2024

Conversation

fjetter
Copy link
Member

@fjetter fjetter commented Nov 7, 2024

we've seen some workers not closing properly in cases where the scheduler died very unusual deaths. We saw that workers, particularly Nannies, wouldn't close for whatever reason without any sound.

For this section in the code it is at least in theory possible to lock up in case the connector (in this case tornado which I generally trust to do these things properly) swallows cancellation attempts completely.

I don't have confidence that this is actually related but in case I'm missing something, let's remove that edge case.

@fjetter fjetter force-pushed the commpool_cancelled_error branch from 342c52e to 4d579a7 Compare November 7, 2024 13:30
Copy link
Contributor

github-actions bot commented Nov 7, 2024

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    25 files  ± 0      25 suites  ±0   10h 16m 57s ⏱️ - 1m 52s
 4 129 tests + 1   4 014 ✅ + 1    110 💤 ±0  5 ❌ ±0 
47 680 runs  +12  45 562 ✅ +13  2 113 💤 ±0  5 ❌  - 1 

For more details on these failures, see this check.

Results for commit 4d579a7. ± Comparison against base commit c38c509.

@fjetter fjetter merged commit ff8b2f4 into dask:main Nov 8, 2024
24 of 30 checks passed
@fjetter fjetter deleted the commpool_cancelled_error branch November 8, 2024 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant