Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Disable parallel cythonize on Windows (GH 30214) #30585

Merged
merged 5 commits into from
Jan 1, 2020

Conversation

Dr-Irv
Copy link
Contributor

@Dr-Irv Dr-Irv commented Dec 31, 2019

Added test in setup.py to check if nthreads is positive and on Windows.

@alimcmaster1 alimcmaster1 added the Windows Windows OS label Dec 31, 2019
setup.py Show resolved Hide resolved
@jbrockmendel jbrockmendel added the Build Library building on various platforms label Dec 31, 2019
Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also enable this for Windows CI? Would be curious to see how much time it saves there

doc/source/whatsnew/v1.0.0.rst Outdated Show resolved Hide resolved
@Dr-Irv
Copy link
Contributor Author

Dr-Irv commented Dec 31, 2019

Can you also enable this for Windows CI? Would be curious to see how much time it saves there

@WillAyd If I "enable" it for Windows CI, then all compiles would run sequentially. So that wouldn't save time (assuming the parallel stuff is working now). What am I missing here?

@WillAyd
Copy link
Member

WillAyd commented Dec 31, 2019 via email

@Dr-Irv
Copy link
Contributor Author

Dr-Irv commented Dec 31, 2019

I meant should add the “-j N” flag to the build command for Windows in the CI folder. We currently don’t pass anything so Cythonization and compilation are sequential

OK, I put that in the last commit

Copy link
Contributor

@topper-123 topper-123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried this solution out, and compilation works fine for me.

Would be interesting to try parallizing on Windows when cython/cython#3262 is solved in a new Cython release.

@Dr-Irv Dr-Irv requested a review from WillAyd December 31, 2019 22:26
@jreback jreback added this to the 1.0 milestone Jan 1, 2020
@jreback jreback merged commit a29cee3 into pandas-dev:master Jan 1, 2020
@jreback
Copy link
Contributor

jreback commented Jan 1, 2020

thanks @Dr-Irv

hweecat pushed a commit to hweecat/pandas that referenced this pull request Jan 1, 2020
@Dr-Irv Dr-Irv deleted the issue30356 branch January 3, 2020 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Library building on various platforms Windows Windows OS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

#30214 (Parallelized Build / CI) caused a build failure for me
6 participants