IpoptMKL: new recipe #1031

dpo · 2020-05-14T01:15:19Z

Is this of interest? There have been a few requests for Ipopt/MKL recently over at Ipopt.jl.

dpo · 2020-05-14T01:24:30Z

Not understanding why irrelevant platforms are being built here...

giordano · 2020-05-14T01:28:04Z

What do you mean by "irrelevant"?

dpo · 2020-05-14T01:32:25Z

Oh, I apologize. I forgot that Windows was in there...

dpo · 2020-05-14T19:42:39Z

I'm quite confused that when I try to build for Windows locally, I get

$ BINARYBUILDER_RUNNER=docker julia --project=../../yggdrasil --color=yes build_tarballs.jl --verbose --debug x86_64-w64-mingw32-libgfortran5-cxx11

ERROR: LoadError: IOError: open: no such file or directory (ENOENT)
Stacktrace:
 [1] open at ./filesystem.jl:79 [inlined]
 [2] sendfile(::String, ::String) at ./file.jl:909
 [3] cp(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:351
 [4] cp at ./file.jl:345 [inlined]
 [5] (::BinaryBuilder.var"#57#61"{Prefix,Windows,Array{String,1}})() at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/Prefix.jl:447
 [6] activate(::BinaryBuilder.var"#57#61"{Prefix,Windows,Array{String,1}}, ::String) at /builddir/build/BUILD/julia/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:915
 [7] setup_dependencies(::Prefix, ::Array{Pkg.Types.PackageSpec,1}, ::Windows) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/Prefix.jl:405
 [8] autobuild(::AbstractString, ::AbstractString, ::VersionNumber, ::Array{#s1079,1} where #s1079<:BinaryBuilder.AbstractSource, ::AbstractString, ::Array{T,1} where T, ::Array{#s1078,1} where #s1078<:Product, ::Array{#s1077,1} where #s1077<:BinaryBuilder.AbstractDependency; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, lazy_artifacts::Bool, meta_json_stream::Any, kwargs::Any) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/AutoBuild.jl:597
 [9] build_tarballs(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any; kwargs::Any) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/AutoBuild.jl:183
 [10] build_tarballs(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any) at /home/mgi.polymtl.ca/doorba/dev/julia/BinaryBuilder.jl/src/AutoBuild.jl:84
 [11] top-level scope at /home/mgi.polymtl.ca/doorba/dev/julia/Yggdrasil/I/IpoptMKL/build_tarballs.jl:62
in expression starting at /home/mgi.polymtl.ca/doorba/dev/julia/Yggdrasil/I/IpoptMKL/build_tarballs.jl:62

Any idea what this error means?

giordano · 2020-05-14T19:44:18Z

Any idea what this error means?

Not on top of my head, I'd go up in the stacktrace, to understand what's going on.

dpo · 2020-05-14T21:25:29Z

Modifying src/Prefix.jl on my Mac as follows:

diff --git a/src/Prefix.jl b/src/Prefix.jl
index c282ced..0264235 100644
--- a/src/Prefix.jl
+++ b/src/Prefix.jl
@@ -454,6 +454,8 @@ function setup_dependencies(prefix::Prefix, dependencies::Vector{PkgSpec}, platf
             # Copy the artifact from the global installation location into this build-specific artifacts collection
             src_path = Pkg.Artifacts.artifact_path(artifact_hash)
             dest_path = joinpath(prefix, "artifacts", basename(src_path))
+            @show name src_path dest_path
+            @show readdir(src_path) readdir(dest_path)
             cp(src_path, dest_path)
 
             # Keep track of our dep paths for later symlinking

gives me:

name = "MKL_jll"
src_path = "/Users/dpo/.julia/artifacts/b096ea2066a0eb73885a511bef6608de3845feae"
dest_path = "/Users/dpo/dev/julia/Yggdrasil/I/IpoptMKL/build/x86_64-w64-mingw32/bkDYE2SI/artifacts/b096ea2066a0eb73885a511bef6608de3845feae"
ERROR: LoadError: SystemError: unable to read directory /Users/dpo/.julia/artifacts/b096ea2066a0eb73885a511bef6608de3845feae: No such file or directory

On another machine, I get a similar error with CompilerSupportLibraries_jll.

giordano · 2020-05-16T22:53:21Z

I think this PR needs JuliaPackaging/BinaryBuilder.jl#786

staticfloat · 2020-05-16T23:03:41Z

Heh, yeah, I just spent like an hour trying to figure out why the new arctic3 installation wasn't installing artifacts properly 😅

dpo · 2020-05-17T20:45:42Z

Thanks, the fix in BinaryBuilder appears to be working.

Unfortunately, I've no idea how to link with MKL on MinGW. Does anyone know? configure keeps complaining that it's not finding the symbol dsyev, which is a LAPACK symbol and should be found in the MKL.

svigerske · 2020-05-19T07:14:53Z

Thanks, the fix in BinaryBuilder appears to be working.

Unfortunately, I've no idea how to link with MKL on MinGW. Does anyone know? configure keeps complaining that it's not finding the symbol dsyev, which is a LAPACK symbol and should be found in the MKL.

Check the config.log for details.

dpo · 2020-05-20T22:30:24Z

Disabling Windows entirely as apparently, the MKL is incompatible with gcc compilers on Windows.

giordano · 2020-05-23T19:02:02Z

Good to go?

dpo · 2020-05-23T19:10:05Z

I think so.

vchuravy · 2020-05-23T21:50:41Z

Should this be loadable? JuliaRegistries/General#15232

dpo · 2020-05-23T22:08:27Z

I just tried on my Mac and it works. Not sure why CI is failing.

(@v1.4) pkg> activate ipopt-mkl
 Activating new environment at `/private/tmp/ipopt-mkl/Project.toml`

(ipopt-mkl) pkg> add https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl
    Cloning git-repo `https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl`
   Updating git-repo `https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl`
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Downloading artifact: IpoptMKL
######################################################################## 100.0%#=#=-#  #             Updating `/private/tmp/ipopt-mkl/Project.toml`
  [6af6c270] + IpoptMKL_jll v3.13.2+0 #master (https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl)
   Updating `/private/tmp/ipopt-mkl/Manifest.toml`
  [ae81ac8f] + ASL_jll v0.1.1+3
  [e66e0078] + CompilerSupportLibraries_jll v0.3.3+0
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+0
  [6af6c270] + IpoptMKL_jll v3.13.2+0 #master (https://github.com/JuliaBinaryWrappers/IpoptMKL_jll.jl)
  [856f044c] + MKL_jll v2020.1.216+0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

julia> using IpoptMKL_jll
[ Info: Precompiling IpoptMKL_jll [6af6c270-9bc2-5f6c-a1c9-61e255f839ff]
Downloading artifact: MKL
######################################################################## 100.0%#=#=-#  #

julia> IpoptMKL_jll.libipopt
"@rpath/libipopt.3.dylib"

julia> IpoptMKL_jll.libipopt_handle
Ptr{Nothing} @0x00007fe4c2ec9610

giordano · 2020-05-23T22:16:03Z

I just tried on my Mac and it works. Not sure why CI is failing.

The registry uses a Linux system

Should this be loadable? JuliaRegistries/General#15232

I fear that's because libmkl_intel_lp64 is not a product, or even worse because we don't run autofix on Linux because of #922

dpo · 2020-05-23T22:20:18Z

Correction: it was picking up another libmkl on my system. I confirm that it doesn't load on macOS either.

giordano · 2020-05-23T22:24:39Z

What error do you get? We might just need to add the lib to the product (and autofix is run on Linux, so that's not the problem)

dpo · 2020-05-23T22:56:04Z

I get

julia> using IpoptMKL_jll
ERROR: InitError: could not load library "/Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib"
dlopen(/Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib, 1): Library not loaded: @rpath/libmkl_intel_lp64.dylib
  Referenced from: /Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib
  Reason: image not found

Shouldn't all MKL libs be products?

shell> otool -L /Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib
/Users/dpo/.julia/artifacts/2143f256a9b8d47426c4cf06b82346d9e8aa2f9b/lib/libipopt.3.dylib:
	@rpath/libipopt.3.dylib (compatibility version 17.0.0, current version 17.2.0)
	@rpath/libmkl_intel_lp64.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_core.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_intel_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libiomp5.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)

giordano · 2020-05-23T23:06:25Z

Shouldn't all MKL libs be products?

Yes, they should... However I think that MKL might have different different libraries on different platforms, which makes thing slightly more complicated. Mind opening a PR?

dpo · 2020-05-23T23:30:31Z

#1075

dpo force-pushed the IpoptMKL branch from 79a278a to ffdb302 Compare May 14, 2020 01:21

dpo closed this May 14, 2020

dpo reopened this May 14, 2020

dpo force-pushed the IpoptMKL branch from ffdb302 to 04c8a1f Compare May 14, 2020 01:34

dpo force-pushed the IpoptMKL branch from 04c8a1f to ef30a77 Compare May 14, 2020 22:40

staticfloat closed this May 16, 2020

staticfloat reopened this May 16, 2020

staticfloat closed this May 16, 2020

staticfloat reopened this May 16, 2020

staticfloat closed this May 16, 2020

staticfloat reopened this May 16, 2020

dpo mentioned this pull request May 18, 2020

Precompiled binaries coin-or/Ipopt#377

Closed

IpoptMKL: new recipe

523c7be

dpo force-pushed the IpoptMKL branch from ef30a77 to 523c7be Compare May 20, 2020 22:29

giordano merged commit a267afb into JuliaPackaging:master May 23, 2020

odow mentioned this pull request Nov 23, 2020

distribute with MKL? jump-dev/Ipopt.jl#6

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IpoptMKL: new recipe #1031

IpoptMKL: new recipe #1031

dpo commented May 14, 2020

dpo commented May 14, 2020

giordano commented May 14, 2020

dpo commented May 14, 2020

dpo commented May 14, 2020

giordano commented May 14, 2020

dpo commented May 14, 2020

giordano commented May 16, 2020

staticfloat commented May 16, 2020

dpo commented May 17, 2020

svigerske commented May 19, 2020

dpo commented May 20, 2020

giordano commented May 23, 2020

dpo commented May 23, 2020

vchuravy commented May 23, 2020 •

edited

Loading

dpo commented May 23, 2020

giordano commented May 23, 2020

dpo commented May 23, 2020

giordano commented May 23, 2020 •

edited

Loading

dpo commented May 23, 2020

giordano commented May 23, 2020

dpo commented May 23, 2020

IpoptMKL: new recipe #1031

IpoptMKL: new recipe #1031

Conversation

dpo commented May 14, 2020

dpo commented May 14, 2020

giordano commented May 14, 2020

dpo commented May 14, 2020

dpo commented May 14, 2020

giordano commented May 14, 2020

dpo commented May 14, 2020

giordano commented May 16, 2020

staticfloat commented May 16, 2020

dpo commented May 17, 2020

svigerske commented May 19, 2020

dpo commented May 20, 2020

giordano commented May 23, 2020

dpo commented May 23, 2020

vchuravy commented May 23, 2020 • edited Loading

dpo commented May 23, 2020

giordano commented May 23, 2020

dpo commented May 23, 2020

giordano commented May 23, 2020 • edited Loading

dpo commented May 23, 2020

giordano commented May 23, 2020

dpo commented May 23, 2020

vchuravy commented May 23, 2020 •

edited

Loading

giordano commented May 23, 2020 •

edited

Loading