do not set CPU affinity on Linux (see #3097)

[stevengj]

See the discussion in #3097 and #1802 as well as this mailing-list thread. Julia should not touch the CPU affinity by default; not only is this a hack that degrades overall system usability to inflate benchmark numbers, but it also prevents people from using taskset manually.

[jakebolewski]

I agree that it is best to not set this by default, maybe we could have a command line switch to enable this?

[JeffBezanson]

We didn't do this to inflate benchmark numbers; we did it to prevent openblas from causing julia -p 2 to use a single core. If we can be sure that won't happen, I'd love to remove this code.

[stevengj]

@JeffBezanson, is this for distro builds of OpenBLAS that don't use NO_AFFINITY=1? But my understanding is that NO_AFFINITY is the default for OpenBLAS, so I'm confused about what was going on in #1070.

Even for OpenBLAS compiled without NO_AFFINITY, we could set ENV["OPENBLAS_MAIN_FREE"]=1 according to the openblas docs to disable affinity.

[stevengj]

@jakebolewski, there is already a command-line switch, called taskset.

[stevengj]

It seems that @wernsaar changed the default in OpenBLAS to NO_AFFINITY after similar complaints from the R community; see OpenMathLib/OpenBLAS#439 and OpenMathLib/OpenBLAS@e52532a. It looks like it was changed in OpenBLAS 0.2.9 or later, so in relatively recent versions of Ubuntu (August 2014 or later) and other distros hopefully the issue went away.

Maybe we could set the OPENBLAS_MAIN_FREE environment variable to make sure things work on systems where OpenBLAS is compiled with affinity? Where is the right place to set this to ensure that it gets set before OpenBLAS is initialized? (OpenBLAS is initialized, via gotoblas_init, when the shared library is loaded, because this function is declared via __attribute__ ((constructor)), so this happens before main starts. So it seems like if we want to change the environment variable we need to do it from our own library constructor, although even this might not work if ld randomizes the library-load order? We could also load OpenBLAS manually via dlopen.) Seems like a PITA, and maybe not worth it since this was fixed upstream.

[JeffBezanson]

If openblas' default is NO_AFFINITY now that should be good enough. I believe we already do dlopen openblas, so we can set the env var too for good measure. We already set OPENBLAS_NUM_THREADS sometimes in client.jl.

[stevengj]

Great, since we dlopen OpenBLAS this is easy. Just added a commit to set OPENBLAS_MAIN_FREE=1 if it is not already set.

[nalimilan]

NO_AFFINITY=1 is no longer the default in OpenBLAS for a few releases. It's not clear whether it was clearly intended or not. Please comment on OpenMathLib/OpenBLAS@912410f#r175499678, as I'm totally unable to develop arguments in favor of this.

[stevengj]

Note that since we still set ENV["OPENBLAS_MAIN_FREE"] = "1", hopefully the NO_AFFINITY flag in OpenBLAS should not affect us.

[nalimilan]

Yes. I just got a report in Fedora about this, but AFAICT it's not related to Julia but to the Fedora OpenBLAS package.

[tomhaber]

Is there still a reason for this behavior? I'm using Julia together with MPI and this overrides whatever MPI tells it to do and there is no way to disable this. I'd be in favor of just using what is assigned to the process.

In applications I typically look at the mask to determine the number of available logical cores assigned to the process.

I'm also confused about how this would interact with "julia -p 2". Does everyone take control of all available cores?