Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build wheels for Python 3.11 #1247

Merged
merged 4 commits into from
Sep 22, 2022

Conversation

arcondello
Copy link
Member

Also switch to manylinux2014.

@arcondello arcondello force-pushed the feature/Python3.11 branch 9 times, most recently from cf23ce0 to 357396b Compare September 16, 2022 21:38
@arcondello
Copy link
Member Author

arcondello commented Sep 16, 2022

python/cpython#29560 introduced a warning when a SpooledTemporaryFile is deleted without being previously closed. Going to need to make a lot of changes to avoid the warning.

@arcondello
Copy link
Member Author

arcondello commented Sep 16, 2022

Also, the time it takes to run unittests on my system has gone from 18s to 60s. Will need to look into the source of the slowdown. Though I imagine the interactions between the performance improvements they made for 3.11 and Cython is to blame. Nevermind, bad measurements.

@arcondello arcondello force-pushed the feature/Python3.11 branch 2 times, most recently from 6923dc9 to d43a4de Compare September 16, 2022 23:56
@codecov-commenter
Copy link

Codecov Report

Merging #1247 (d43a4de) into main (00fed80) will increase coverage by 0.19%.
The diff coverage is 94.73%.

❗ Current head d43a4de differs from pull request most recent head ee4bc52. Consider uploading reports for the commit ee4bc52 to get more accurate results

@@            Coverage Diff             @@
##             main    #1247      +/-   ##
==========================================
+ Coverage   94.89%   95.09%   +0.19%     
==========================================
  Files          83       83              
  Lines        8208     8214       +6     
==========================================
+ Hits         7789     7811      +22     
+ Misses        419      403      -16     
Impacted Files Coverage Δ
dimod/serialization/fileview.py 92.41% <90.00%> (ø)
dimod/constrained.py 94.73% <100.00%> (+0.05%) ⬆️
dimod/binary/binary_quadratic_model.py 93.26% <0.00%> (+0.33%) ⬆️
dimod/quadratic/quadratic_model.py 92.77% <0.00%> (+0.57%) ⬆️
dimod/binary/vartypeview.py 97.81% <0.00%> (+1.09%) ⬆️
dimod/binary/pybqm.py 96.78% <0.00%> (+1.20%) ⬆️
dimod/typing.py 97.72% <0.00%> (+11.36%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@arcondello arcondello force-pushed the feature/Python3.11 branch 4 times, most recently from 8700f8e to 1f5e24a Compare September 19, 2022 15:44
@arcondello arcondello marked this pull request as ready for review September 19, 2022 16:01
@arcondello arcondello requested a review from randomir September 19, 2022 16:01
Copy link
Member

@randomir randomir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -404,17 +392,10 @@ workflows:
- build-linux: &build
matrix:
parameters:
python-version: &python-versions [3.7.9, 3.8.9, 3.9.4, 3.10.0]
python-version: [3.7.9, 3.8.9, 3.9.4, 3.10.0, 3.11.0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think 3.11 is released yet. Oh, I see, new cibuildwheel has an alias to rc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, for consistency it would make sense to use 3.11.0rc1 here as well.

@@ -450,7 +431,7 @@ workflows:
- build-windows
matrix:
parameters:
python-version: *python-versions
python-version: [3.7.9, 3.8.9, 3.9.4, 3.10.0] # no 3.11 with nuget yet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use DockerHub's Windows Python image (instead of installing python with nuget -- should speed-up and simplify things), like 3.11.0rc2-windowsservercore-ltsc2022? Actually, using 3.11-rc or 3.11.0rc1 on windows executor should suffice (as these are multi-platform images, just like our ocean-docker is 😆).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea! Will look into it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this has turned into a hellscape of windows syntax. Specifically flavours of this. I have run out of patience so I think we should just skip 3.11 tests for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can try with shell: bash -eo pipefail.

Or yes, just skip 3.11 tests for now.

Copy link
Member Author

@arcondello arcondello Sep 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I have it working. Now the issue is that it takes 4+min to download the docker image! I tried seeing if I can speed it up with docker save/docker load but those are also stupidly slow.
For comparison the nuget install was ~20s.
Anyway, I am sick of windows so will think more about it tomorrow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's awful. 🤢 😱

I can't understand why CircleCI doesn't cache those 2GB+ docker images. I bet GitHub Actions start Windows images in seconds 😝

So, I retract my initial idea of using Windows images.

features:
- Build wheels for Python 3.11.
upgrade:
- Build ``manylinux2014`` wheels. Previously we built ``manylinux2010``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity: manylinux2014 wheels require pip >= 19.3 that comes bundled with python 3.7.8+, 3.8.4+, 3.9.0+. python ~= 3.7.0,<3.7.3 with older pip hence will not pull these images without pip upgrade.

This is a slight annoyance, since upgrading pip is generally desirable (and pip itself is very aggressive in its prods to upgrade). Additionally, we'll drop 3.7 support in not too distant future (it EOLs 2023-06).

@arcondello arcondello force-pushed the feature/Python3.11 branch 7 times, most recently from dc469bb to 8195f68 Compare September 21, 2022 22:10
@arcondello arcondello changed the base branch from main to 0.12.0 September 22, 2022 20:01
@arcondello arcondello merged commit e54b180 into dwavesystems:0.12.0 Sep 22, 2022
@arcondello arcondello deleted the feature/Python3.11 branch September 22, 2022 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants