Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce nthreads = 1 for single threaded lib #431

Merged
merged 3 commits into from
Apr 22, 2024

Conversation

lu1and10
Copy link
Member

@lu1and10 lu1and10 commented Apr 19, 2024

Enforce nthreads = 1 for single threaded lib, to improve the case that single threaded finufft lib running with nthreads != 1 gives inf/nan error flatironinstitute/jax-finufft#73 and #430.

@lu1and10 lu1and10 requested review from blackwer and ahbarnett April 19, 2024 14:58
@dfm
Copy link
Contributor

dfm commented Apr 19, 2024

I'm looking at this now too, and another option would be to use the same approach as the one in spreadinterp.cpp, avoiding the extra ifdef:

int maxnthr = MY_OMP_GET_MAX_THREADS();
if (opts.nthreads>0) // user override up to max avail
maxnthr = min(maxnthr,opts.nthreads);

I don't have any argument for why one would be better than the other, but could be good to be consistent!

Edited to confirm that this does seem to fix flatironinstitute/jax-finufft#73.

@lu1and10
Copy link
Member Author

lu1and10 commented Apr 19, 2024

I'm looking at this now too, and another option would be to use the same approach as the one in spreadinterp.cpp, avoiding the extra ifdef:

int maxnthr = MY_OMP_GET_MAX_THREADS();
if (opts.nthreads>0) // user override up to max avail
maxnthr = min(maxnthr,opts.nthreads);

I don't have any argument for why one would be better than the other, but could be good to be consistent!

Edited to confirm that this does seem to fix flatironinstitute/jax-finufft#73.

Do you mean these lines https://github.com/lu1and10/finufft/blob/9744d14b07b2d25bb9e5326d549bb1d7971a7f88/src/spreadinterp.cpp#L269-L271
right, it's better consistent. we can do that always cap num threads with MY_OMP_GET_MAX_THREADS(), I guess we should still warn user that user's nthreads input is not used in case nthr = min(nthr,p->opts.nthreads) chooses nthr not opts.nthreads?

@lu1and10
Copy link
Member Author

lu1and10 commented Apr 19, 2024

the change here does not cap nthreads to omp_get_max_threads if omp is enabled, this is the same as the original code

nthr = p->opts.nthreads; // user override (no limit or check)
.
It only cap nthreads to 1 if omp is not enabled. @dfm suggests we could be consistent with spreader as in
int maxnthr = MY_OMP_GET_MAX_THREADS();
if (opts.nthreads>0) // user override up to max avail
maxnthr = min(maxnthr,opts.nthreads);
to always cap nthreads to omp_get_max_threads for both omp enabled or not. @ahbarnett is there a reason the current code does not cap nthreads to omp_get_max_threads? It seems that nthreads also controls the num of threads for fft.

@blackwer
Copy link
Member

My vote is for consistency. Always cap threads at the requested thread count. If it's single a single threaded build: 1, otherwise opts.nthreads. I'm ambivalent about the warning, but it seems reasonable.

@lu1and10
Copy link
Member Author

My vote is for consistency. Always cap threads at the requested thread count. If it's single a single threaded build: 1, otherwise opts.nthreads. I'm ambivalent about the warning, but it seems reasonable.

I see, let's see what @ahbarnett 's thought about whether to use omp_get_max_threads or user requested threads count to cap nthreads.

The warning could be better, English is not my native.... Concise and pretty warning message is appreciated.

@ahbarnett
Copy link
Collaborator

Thanks for parsing the logic here. I agree we should cap to omp_get_max_threads() and that being set to 1 indicates single-thread mode without invoking #ifdefs. So I like Dan's idea of following what I did in spreadinterp, but adding the warning. I'm going to try and push to your PR, Libin.

@lu1and10
Copy link
Member Author

Thanks for parsing the logic here. I agree we should cap to omp_get_max_threads() and that being set to 1 indicates single-thread mode without invoking #ifdefs. So I like Dan's idea of following what I did in spreadinterp, but adding the warning. I'm going to try and push to your PR, Libin.

Thanks @ahbarnett, not sure you can push to my fork, added you to the fork collaborator, maybe we should use the official finufft repo creating a new branch for PR later on.

Copy link
Collaborator

@ahbarnett ahbarnett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was good and respected the "unlimited thread override" the user could do. But based on Libin + Dan + Robert suggestions, I changed it to always limit to maxnthr given by OMP ( = 1 if single-threaded build, as per defs.h header), matching spreadinterp, and preventing JAX crashes.

@ahbarnett
Copy link
Collaborator

Happy for Libin etc to pull in, assuming CI good. Thanks!

@lu1and10 lu1and10 closed this Apr 22, 2024
@lu1and10 lu1and10 reopened this Apr 22, 2024
@lu1and10 lu1and10 merged commit 0fd60a0 into flatironinstitute:master Apr 22, 2024
16 of 17 checks passed
@ahbarnett
Copy link
Collaborator

I forgot to update CHANGELOG and docs. Will do it in master. We should always do this for changes to behavior (what if a user relied on setting nthr > get_max_omp_threads() ?)

ahbarnett added a commit that referenced this pull request May 8, 2024
… ifndef _OPENMP hard nthr=1 override; warnings at the finufft level only; binsort controlled by spopts.sort_threads matching docs
@lu1and10 lu1and10 deleted the omp-warning branch June 25, 2024 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants