Update UCX tests for new handshake step #4036
Conversation
Recently Dask added a handshake step to communicate information like Python versions available and feature supported (compressors, out-of-band pickling, etc.). This causes some UCX test failures in their current form. So this updates them to fix that issue.
|
Thank you for this @jakirkham . This looks good to me. Will merge on passed tests. |
Not sure why this test started segfaulting for me. However we have had some history with segfault issues in these tests previously. So go ahead and just disable this one with a note as to why.
| pytest.param( | ||
| lambda cudf: cudf.DataFrame({"a": ["a"]}).head(0), | ||
| marks=pytest.mark.skip( | ||
| reason="This test segfaults for some reason. So skip running it entirely." | ||
| ), | ||
| ), |
There was a problem hiding this comment.
There was a problem hiding this comment.
Hmm this is news to me but is likely problematic. We'll have to poke around a little bit.
There was a problem hiding this comment.
There is also another case in this test (shown below) that has been marked as failing for a while as well. Not sure if they are related.
distributed/distributed/comm/tests/test_ucx.py
Lines 180 to 185 in 1747ae1
There was a problem hiding this comment.
I'm curious, did this fail before the recent handshake PR?
There was a problem hiding this comment.
I haven't seen it in UCX-Py's CI, but did run into it locally. Maybe there's something more subtle at play like environment differences. In any event, I don't think it is handshake related.
|
Merged to fix ucx-py CI. Hope that is ok 🙂 |
Follow-up to PR ( #4019 )
Recently Dask added a handshake step to communicate information like Python versions available and feature supported (compressors, out-of-band pickling, etc.). This causes some UCX test failures in their current form. So this updates them to fix that issue.
cc @quasiben @pentschev @madsbk @mrocklin