Shipping MKL with julia now that USE_GPL_LIBS Makefile flag exists

[hiccup7]

In the previous discussion entitled "Shipping MKL with julia" #4272
the conclusion was that this was not legal with GPL code inclusion. Now, this limitation has been removed: #10870

I propose that installers on the julia downloads page are offered in both OpenBLAS (USE_GPL_LIBS=1) and MKL (USE_GPL_LIBS=0) variants. It may not be necessary to offer the MKL variant for Linux since there is a free option for developers to build from source:
https://software.intel.com/en-us/qualify-for-free-software/opensourcecontributor

As a DSP developer, the BLAS library dominates my speed experience with julia because it is the only way to get multi-threaded SIMD convolutions presently. I and others have submitted problem reports about OpenBLAS being several times slower than MKL: https://github.com/xianyi/OpenBLAS/issues
However, we cannot expect a small group of hobby volunteers to resolve these problems in a timely manner. Shipping julia with MKL would allow developers and users to experience julia's speed while we wait (maybe 6-12 months) for OpenBLAS issues to be resolved. When and if OpenBLAS is at least as fast as MKL, then MKL could be dropped from julia's download page.

Since Intel has offered MKL licenses to the julia team for free, I think it would be preferable to ship julia with MKL than to ask Windows and OSX julia users to all purchase MKL and build from source.

For comparison, I know of 3 Python Windows binary distributions that include MKL for free. This makes Python the fastest free HPC language for Windows at present. I propose that shipping julia with MKL will get a lot of users to switch from Python to Julia, which will increase contributions to julia and improve the rate of julia development.

Personally, I would like to convert the code for the company I work for from MATLAB to julia, but building from source is a deal-breaker for me. If I have to wait a year for OpenBLAS' problems to be resolved, it would be better to convert the MATLAB code to Python instead. It is a big effort for me to port this code, so I only want to do it once (and I would have trouble convincing my employer to pay for two portings). In case anyone thinks a year is too pessimistic for resolving OpenBLAS problems, note that the first thread above was from 1.5 years ago when forum members naively estimated that they would fix OpenBLAS by now.

I would really like julia v0.3.8 with MKL!

[ViralBShah]

Intel now has a new policy, which includes access to licenses for open source:

https://software.intel.com/en-us/qualify-for-free-software

I really would like to make this happen, but it is a non-trivial amount of work. I would love it if someone wants to take this up for 0.3.8. The USE_GPL_LIBRARIES flag needs backporting, but that should be easy. Realistically, I think targeting 0.4 with MKL support is a better option.

[tkelman]

I want to tag 0.3.8 before the end of the month. This won't happen by then.

Julia only partially works with MKL when built with Intel compilers on Linux and OSX, there are still test failures. Building Julia with Intel compilers on Windows has not been tried for over a year, as far as I'm aware. This will take major effort, and requires contributions from people who are willing to help with source builds and patches.

[tkelman]

#2167 is a prerequisite for using ifort and/or MKL on Windows.

[tkelman]

Or we might be able to get away with some refactoring via blasfunc here

julia/base/util.jl

Lines 129 to 135 in c56c03f

	if blas_vendor() == :openblas64
	blasfunc(x) = string(x)*"64_"
	openblas_get_config() = strip(bytestring( ccall((:openblas_get_config64_, Base.libblas_name), Ptr{UInt8}, () )))
	else
	blasfunc(x) = string(x)
	openblas_get_config() = strip(bytestring( ccall((:openblas_get_config, Base.libblas_name), Ptr{UInt8}, () )))
	end

to handle the underscores in a single place for the time being until fcall gets implemented. That won't help with ARPACK or openspecfun when built with ifort (assuming the respective build systems can be made to jump through all necessary hoops), but it might be better than nothing.

Note that in the meantime, it should be possible to use ccall with more or less direct copies of the existing BLAS/LAPACK wrapper functions, but change the library name to the MKL dll and remove trailing underscores. This might work without needing any patches to base Julia. A * B would still call OpenBLAS by default, but you could add an MKL module where MKL.gemm! would call into MKL.

[ScottPJones]

This would be very good... when I built Julia for the first time last night, I was surprised out how many warnings came from building OpenBLAS... (and some looked like they could actually cause problems).
What alternatives for the other GPL parts of Julia's Base exist? Is there one for Sparsesuite?
It turns out that one of the developers would like to be able to use it (or something like it), but can work around it, if necessary.

[tkelman]

I was surprised out how many warnings came from building OpenBLAS... (and some looked like they could actually cause problems).

I think many of those are in code that openblas is intentionally using unchanged from upstream netlib sources, but maybe not all.

Is there one for Sparsesuite?

There are a variety of libraries for sparse linear algebra that could be used instead of SuiteSparse, however they are far from drop-in API-compatible replacements.

Julia bindings to MKL's Pardiso sparse solver could fill the same roles as we use SuiteSparse for, but need to be written by someone.

It turns out that one of the developers would like to be able to use it (or something like it), but can work around it, if necessary.

I don't understand what you mean by that.

And regarding actually shipping binaries that use MKL (and/or Intel compilers), we would also have to set up the necessary infrastructure on our buildbots. I don't think any of this is well-tested enough for us to endorse via official binaries at this point, but it's worth working towards.

[ScottPJones]

@tkelman The reason I originally started to look at Julia was because one of the developers had heard in might be good for what they wanted to do, and so I started to evaluate it, and realized that it would be rather good for a number of things the rest of us were doing (not math stuff ;-) ). I then found out about the issue with the 3 GPLed libraries, which luckily has been solved by setting USE_GPL_LIBS to 0.
Then I found out that the developer originally interested in Julia would like to do some of the things provided by Sparsesuite... if I have to create Julia bindings myself to this MKL Pardiso sparse solver for him to proceed, I will, and of course put it up on GitHub, with MIT license.

[tkelman]

Great. I'm just missing context on who you mean by "one of the developers."

Creating an MKLPardiso.jl package would be a good place to start. The API documentation is at https://software.intel.com/en-us/node/470282 and is pretty comprehensive (double-check the licenses, but I think going from just the API documentation you should be in the clear).

[tkelman]

Be aware when wrapping Pardiso that there are separate implementations - the version available in MKL differs in API from the original algorithm developers' newer versions, see http://software.intel.com/en-us/articles/summary-of-api-differences-between-intel-mkl-pardiso-and-university-of-basel-pardiso-400 for some of the differences. There may be interest in wrapping the non-MKL implementation from http://pardiso-project.org/, but binaries of that version of the library will not be redistributable.

[ViralBShah]

MKL can do chol, lu, and ldlt for sparse using Pardiso. That still leaves the sparse QR.

[ScottPJones]

@tkelman Oh, that is one of the developers at a little startup I'm consulting for currently... lots of fun!

[hiccup7]

Would it be helpful to split this issue into sub-issues to ease management? Perhaps an umbrella issue for getting MKL to work, and another issue for support with the build bots? The umbrella issue could be considered a bug fix and have sub-issues for each OS. A bug fix for an OS that used to work would then be considered a regression bug fix.

[ScottPJones]

@tkelman I'll have to pin down exactly what features the developer was hoping to use, so I pin down what possible alternatives there are (whether I have to add the bindings myself or not)

[tkelman]

That still leaves the sparse QR.

qr_mumps is LGPL http://buttari.perso.enseeiht.fr/qr_mumps/ - decent candidate for wrapping in an external package

or quern is public domain https://www.cs.ubc.ca/~rbridson/quern/, and simple enough that it could probably be ported to pure Julia without much trouble

[hiccup7]

Did this thread drop off the radar because of the Julia v0.3.8 release effort?

[tkelman]

Not really. I've described above the reasons this is not straightforward, see #10969 (comment) - there are a number of separate issues open about build/test problems with MKL and/or Intel compilers on Linux and OSX. I don't have Intel compilers or MKL on Windows, and trying to compile Windows Julia with them is not something that anyone has tried in well over a year, to the best of my knowledge.

If you have a particular function in MKL that you know is performance-critical to your work that you would like to use, it should be quite simple to write a ccall wrapper for it, without needing to modify any of the way Julia is built.

[PallHaraldsson]

[EDIT: You can ignore my license discussions below, as Karpinski clarified “Excluded License” in not a restriction, at least to proprietary MKL and similar situations.]

Regarding LGPL in general and "qr_mumps is LGPL http://buttari.perso.enseeiht.fr/qr_mumps/ - decent candidate for wrapping in an external package", it can't be used. Looking at the other thread it seems the make-flag is for removing only GPL, not LGPL (there was at least one library with it) but Intel has problem with LGPL and other licenses.

While I know, say, that MPL is copyleft, it is an even weaker form (file based) than LGPL and they even forbid that one. Am I not reading reading the quote below right? I would have thought they where ok with dynamically linked LGPL (then forbidden "reverse engineering" is not a problem) but I'm not sure. Maybe the only forbid statically linked LGPL (not my reading), as most only have a problem with that. I'm not sure what Julia does.

https://software.intel.com/sites/default/files/managed/c0/5f/Master-EULA-for-Intel-Sw-Development-Products-March-2015.pdf

“Excluded License” means a license that requires, as a condition of use, modification, or distribution, that the licensed software or other software incorporated into, derived from or distributed with such software (a) be disclosed or distributed in Source Code form; (b) be licensed by the user to third parties for the purpose of making and/or distributing derivative works; or (c) be redistributable at no charge. Excluded Licenses include, without limitation, licenses that license or distribute software under any of the following licenses or distribution models, or licenses or distribution models substantially similar to any of the following: (a) GNU’s General Public License (GPL) or Lesser/Library GPL (LGPL), (b) the Artistic License (e.g., PERL), (c) the Mozilla Public License, (d) the Netscape Public License, (e) the Sun Community Source License (SCSL), (f) the Sun Industry Source License (SISL), and (g) the Common Public License (CPL).

[ihnorton]

The point of the LGPL is that (when dynamically linked) it only applies to
the specific code in question, not to the whole project.

On Wed, May 6, 2015 at 11:10 AM, Páll Haraldsson [email protected]
wrote:

Regarding LGPL in general and "qr_mumps is LGPL
http://buttari.perso.enseeiht.fr/qr_mumps/ - decent candidate for
wrapping in an external package", it can't be used. Looking at the other
thread it seems the make-flag is for removing only GPL, not LGPL (there was
at least one library with it) but Intel has problem with LGPL and other
licenses.

While I know, say, that MPL is copyleft, it is an even weaker form (file
based) than LGPL and they even forbid that one. Am I not reading reading
the quote below right? I would have thought they where ok with dynamically
linked LGPL (then forbidden "reverse engineering" is not a problem) but I'm
not sure. Maybe the only forbid statically linked LGPL (not my reading), as
most only have a problem with that. I'm not sure what Julia does.

https://software.intel.com/sites/default/files/managed/c0/5f/Master-EULA-for-Intel-Sw-Development-Products-March-2015.pdf

“Excluded License” means a license that requires, as a condition of use,
modification, or distribution,
that the licensed software or other software incorporated into, derived
from or distributed with such
software (a) be disclosed or distributed in Source Code form; (b) be
licensed by the user to third
parties for the purpose of making and/or distributing derivative works; or
(c) be redistributable at no
charge. Excluded Licenses include, without limitation, licenses that
license or distribute software
under any of the following licenses or distribution models, or licenses or
distribution models
substantially similar to any of the following: (a) GNU’s General Public
License (GPL) or
Lesser/Library GPL (LGPL), (b) the Artistic License (e.g., PERL), (c) the
Mozilla Public License, (d)
the Netscape Public License, (e) the Sun Community Source License (SCSL),
(f) the Sun Industry
Source License (SISL), and (g) the Common Public License (CPL).

—
Reply to this email directly or view it on GitHub
#10969 (comment).

[PallHaraldsson]

Yes, I know, I just wanted to make sure that Intel really allows that. Maybe they do (or should). I am not a lawyer but it seems Intel can and does exclude LGPL code. Are you saying "Julia as a whole" will not be GPL and then can call MKL (and that not the LGPL code will do the calling)?

There is no similar "LGPL as a whole" concept (statically linking would make "GPL as a whole"). Why would they (their license) exclude the LGPL, not just the GPL? I assume you and I am right, I just giving head-up in case..

Hopefully it's not only FUD on my part..

[StefanKarpinski]

@PallHaraldsson, you've quoted the definition of an "Excluded License" but not the part of the license where the term is used, namely in section 4, "LICENSE CONDITIONS":

D. Except as expressly provided in this Agreement, You may NOT: ... (xi) modify, create a
derivative work, link, or distribute the Materials so that any part of it becomes subject to an Excluded
License.

Distributing libjulia with both MKL and an LGPL library does not cause MKL to become subject to the LGPL, so this clause is not violated (if it were GPL it would violate this clause). The effect of this clause is to make any violation of a viral license also a violation of Intel's license. Normally, if you ship something linked against a proprietary library and a GPL library, only the copyright holder of the GPL library can sue for copyright infringement. With this clause if you ship MKL linked together with a GPL library, Intel can also sue you for copyright infringement since you are not in compliance with the terms of their license.

[StefanKarpinski]

In other words: libraries under "Exclude Licenses" are fine to distribute along with Intel libraries as long as you don't combine them in such a way that the Intel libraries would "become subject" to those licenses.

[ScottPJones]

@StefanKarpinski That's good to know... otherwise this could have affected being able to use @stevengj DecFP package (which I very much want to do).

[PallHaraldsson]

2015-05-06 18:05 GMT+00:00 Scott P. Jones [email protected]:

@StefanKarpinski https://github.com/StefanKarpinski That's good to
know... otherwise this could have affected being able to use @stevengj
https://github.com/stevengj DecFP package (which I very much want to
do).

Actually not (I think), I believe DecFP is under a clear (permissive) open
source license.

MKL is proprietary (unlike some other Intel code) and would in no way work
with GPL (GPL would forbid as well as vice versa). They however have this:

https://software.intel.com/en-us/qualify-for-free-software/opensourcecontributor

that applies (assuming LGPL is not an issue). It doesn't make MKL open
source or any less proprietary. I didn't read to carefully, it seems Julia
could distribute, but I'm not sure if you could then redistribute (or
use).. (unless then maybe for non-commercial). Just didn't check..

https://software.intel.com/en-us/articles/non-commercial-software-development

https://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq

[ScottPJones]

@PallHaraldsson DecFP's license is one thing... but it is using an Intel decimal floating point library, just like the one being discussed here, so that was my concern. If there are any issues, that's another reason to move to using the decNumber code, which is under the ICU license.

[tkelman]

Back to the point of this issue, it's worth noting that the "open source contributor" license program at https://software.intel.com/en-us/qualify-for-free-software/opensourcecontributor appears to only have the Linux versions of the Intel compilers available. OS X or Windows would have to be done through some other channel.

[hiccup7]

With the Linux version, can you cross-compile for OSX and Windows targets?

[ihnorton]

No. At one point I believe Viral had licenses for all platforms (see the other thread about this). I'm not sure it that is still the case.

[ViralBShah]

I think those may have expired, but I have received a new license from Intel just a week ago. Also, they now have an official open source policy - so it should be straightforward to get licenses.

[tkelman]

@ViralBShah The question is specifically, for which platforms. Unless you're reading something different than we are, the "official open source policy" appears to only cover Linux versions.

[ViralBShah]

I just assumed they would have a sane policy if they rethought their open source policy.

They have changed everything and it just seems like a real pain to figure out how to use the new licence I received with the new website. I will try to get licences for the other platforms.

[tkelman]

Leaving a link here to the new Pardiso.jl package by @KristofferC for those interested in potential replacements for SuiteSparse: https://github.com/KristofferC/Pardiso.jl - it doesn't support the MKL version of Pardiso yet but probably could be made to do so without a huge amount of work.

[KristofferC]

Please note that the package is not properly tested yet since I started with this today. Feedback is welcome :)

[ViralBShah]

Thanks. This is great to know. @KristofferC At some point, you may consider having this package in JuliaSparse, if you don't mind. Hopefully that will lead to more collaborations and eyeballs.

[tkelman]

+1 to that idea of eventually having the package in JuliaSparse once it's up and running. Here's some autotools code that may or may not be a useful reference for some symbols to check for to tell apart the MKL version of Pardiso from the newer versions: https://github.com/coin-or/Ipopt/blob/769c7240a1ca17dd1ba72cc8f7f52654bb57388b/Ipopt/configure.ac#L291-L365

[KristofferC]

I updated the package to an API which felt much more sensible than the last. I could put it into JuliaSparse but it would maybe be nice if it was a bit more tested first.

For example, I tried to run it on my other computer where I have MKL blas and run into that PARDISO can't find the dgemv_64_ symbol.

This issue seem relevant:
#4923

[ViralBShah]

Yeah, we need to be able to disable that when using MKL.

[tkelman]

#8734 only applies when Julia is built with openblas. By

where I have MKL blas

do you mean Julia is built against MKL?

[KristofferC]

Yes, exactly. To be honest, I haven't had time to try a lot of stuff. Maybe with MKL you have to dlopen something extra. I remember when you link with MKL BLAS you need to also link some extra library.

I am away from the MKL computer for a couple of days but when I come back I will do some more thourogh research.

[KristofferC]

https://github.com/KristofferC/Pardiso.jl

now works with both MKL Pardiso and Pardiso v5.0. If there is interest it could be moved to JuliaSparse. Maybe I should put it in METADATA too?

[tkelman]

Awesome work! I think it could be moved to JuliaSparse now, though it looks like you could still use some assistance to get things working on non-Linux platforms. May want to get at least OSX working before registering?

[KristofferC]

Yeah, any kind of help with testing would be appreciated. Regarding OSX/Windows, I have very little experience in loading libraries for these systems.

It's a bit annoying when you deal with commercial software that there isn't any easy way to set up automatic testing,

[ViralBShah]

I don't know who will have MKL on OS X. Best to register, and hope someone will come along.

[KristofferC]

I think I will register it soon. It works well for me on two different computers, one with both MKL and 5.0 installed and one with only 5.0 so it seems kinda stable on linux at least.

[ViralBShah]

JuliaPro provides this should you want it, as a free download. I doubt the official Julia binaries will have this in the foreseeable future.