Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: 24.12 nightly wheel tests failing with SSL certificate issues #4818

Closed
2 tasks done
jameslamb opened this issue Dec 9, 2024 · 3 comments
Closed
2 tasks done
Labels
? - Needs Triage Need team to review and classify bug Something isn't working

Comments

@jameslamb
Copy link
Member

Version

24.12

Which installation method(s) does this occur on?

No response

Describe the bug.

Many nightly wheel tests are failing on 24.12. I see dozens of failures, in two broad categories.

Stuff like this:

FAILED python/cugraph/cugraph/tests/traversal/test_sssp.py::test_sssp_data_type_conversion[1-graph_file0] - KeyError: "results for {'algo': 'single_source_dijkstra_path_length', 'graph_dataset': 'karate', 'graph_directed': 'True', 'source': '1', 'test': 'data_type_conversion', 'resultset_name': 'traversal'} not found"

And stuff like this:

FAILED python/cugraph/cugraph/tests/utils/test_dataset.py::test_download[dataset0] - urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)>

(build link)

These failures only occur on the CUDA 11.8.0, Python 3.10 wheels test jobs.

Minimum reproducible example

See CI links above.

Relevant log output

See CI links above.

Environment details

See CI links above.

Other/Misc.

Because they looked network-related, I did try manually re-running these failed jobs... they failed in exactly the same way.

I know that cugraph tests do some downloading of externally-hosted datasets... maybe one of those sites hosting that data had an outage or changed some configuration?

Code of Conduct

  • I agree to follow cuGraph's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
@jameslamb jameslamb added ? - Needs Triage Need team to review and classify bug Something isn't working labels Dec 9, 2024
@rlratzel
Copy link
Contributor

rlratzel commented Dec 9, 2024

I'm wondering if the solution here is relevant for these SSL errors.

@nv-rliu nv-rliu mentioned this issue Dec 10, 2024
raydouglass pushed a commit that referenced this issue Dec 11, 2024
Addresses errors in `wheel-tests` seen
[here](#4818)

Installs SSL certificates with `certifi` as outlined in this issue
[here](rapidsai/build-infra#56)
jameslamb pushed a commit to alexbarghi-nv/cugraph that referenced this issue Dec 12, 2024
Addresses errors in `wheel-tests` seen
[here](rapidsai#4818)

Installs SSL certificates with `certifi` as outlined in this issue
[here](rapidsai/build-infra#56)
@nv-rliu
Copy link
Contributor

nv-rliu commented Dec 17, 2024

Can this be closed now?

@jameslamb
Copy link
Member Author

I think so, yes.

The latest 24.12 test run did fail: https://github.com/rapidsai/cugraph/actions/runs/12349459995

As did the latest 25.02 test run: https://github.com/rapidsai/cugraph/actions/runs/12372518014

Neither looks like this exact issue though. Let's close it. You may want to watch rapidsai/ci-imgs#218. Once it goes out, it might be possible to rever the changes from #4825 or some subset of them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
? - Needs Triage Need team to review and classify bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants