Automated binary installs

[odow]

Leaving breadcrumbs to how we automatically install the Gurobi binary on linux (users still need to create and grbgetkey a license):

Create an artifacts file
https://github.com/jump-dev/Gurobi.jl/blob/master/Artifacts.toml

Here's the script to create the hashes
jump-dev/Gurobi.jl#437 (comment)

Don't write a deps file if not needed
https://github.com/jump-dev/Gurobi.jl/blob/e1a163acfa8aca98059bbef43c0aefe0f540591d/deps/build.jl#L172-L175

Load the artifact
https://github.com/jump-dev/Gurobi.jl/blob/e1a163acfa8aca98059bbef43c0aefe0f540591d/src/Gurobi.jl#L16-L23

[lschork2]

Thank you for the detailed instructions. I have added this for Linux and Mac. Two questions:

(1) Do I understand correctly that only tar files can be extracted, but no zip files? So if we wanted to do it for Windows, we would need to release the COPT package as tar.gz also on Windows?

(2) When we update the Artifacts.toml to a new COPT version and users of the old version run Pkg.update("COPT"), would this download the new version?

[odow]

1. Yeah, it has to be a .tar.gz file, even on Windows :(
2. We need to create tagged releases, which I'm about to reply to Qi about/open a new issue. But yes, if you update Artifacts, then tag a new version, when people run update it will download the new version. It's all very smooth!

[odow]

The binaries are working great on linux and Mac: #6

If you can add binaries for Windows, then installing will be a breeze, and we can remove all the hacky stuff in /deps.

[COPT-Public]

We made a tar.gz package for Windows and adjust the code. When we test it, the installation stuck at loading the dll. Full error messages are:

Precompiling project...
  ✗ COPT
  0 dependencies successfully precompiled in 1 seconds (38 already precompiled)
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

julia> Pkg.precompile()
Precompiling project...
  ✗ COPT
  0 dependencies successfully precompiled in 1 seconds (38 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f]

Failed to precompile COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f] to C:\Users\Huangfu\.julia\compiled\v1.7\COPT\jl_22F.tmp.
ERROR: LoadError: could not load library "C:\Users\Huangfu\.julia\artifacts\ac107ec0930ed47ecbe87665f0aeed39be40c904\copt40\bin\copt.dll"
Access is denied.

I tested by manually clicking to grant it access right, and then it works. I repeated by overwriting the dll file by the one I extracted from the tar.gz, and it also works.

Do you have any experience regarding fixing this issue? @odow

[odow]

What are the permissions on the dll before you tar it? And permissions on the tar? Julia doesn't do anything special, it just extracts the tar, keeping whatever permissions were there in the first place. I don't have a windows machine to test, unfortunately.

…

On Fri, 29 Apr 2022, 9:11 PM COPT-Public, ***@***.***> wrote: We made a tar.gz package for Windows and adjust the code. When we test it, the installation stuck at loading the dll. Full error messages are: Precompiling project... ✗ COPT 0 dependencies successfully precompiled in 1 seconds (38 already precompiled) 1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package julia> Pkg.precompile() Precompiling project... ✗ COPT 0 dependencies successfully precompiled in 1 seconds (38 already precompiled) ERROR: The following 1 direct dependency failed to precompile: COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f] Failed to precompile COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f] to C:\Users\Huangfu\.julia\compiled\v1.7\COPT\jl_22F.tmp. ERROR: LoadError: could not load library "C:\Users\Huangfu\.julia\artifacts\ac107ec0930ed47ecbe87665f0aeed39be40c904\copt40\bin\copt.dll" Access is denied. I tested by manually clicking to grant it access right, and then it works. I repeated by overwriting the dll file by the one I extracted from the tar.gz, and it also works. Do you have any experience regarding fixing this issue? @odow <https://github.com/odow> — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6MQJP34JLGRUPLQXW6DXTVHORUDANCNFSM5UN7SIUQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[COPT-Public]

It was created using the tar tool provided by Windows. If unzip it on Windows manually, things work fine. I am afraid that it may take much long to get it fixed as it seems no one has direct experiences regarding this new issue. Is there any other solver who provides automatic download for Windows that we can learn from?

[odow]

Here's the HiGHS binary for Windows:

https://github.com/JuliaBinaryWrappers/HiGHS_jll.jl/releases/download/HiGHS-v1.2.2%2B0/HiGHS.v1.2.2.x86_64-w64-mingw32-cxx11.tar.gz

https://github.com/JuliaBinaryWrappers/HiGHS_jll.jl/releases/tag/HiGHS-v1.2.2%2B0

If you're still having trouble, email me the tarball, and I'll take a look.

[COPT-Public]

I am a bit puzzled by the workflow of HiGHS regarding the Windows automatic build.
The tarball of our Windows version is online at https://pub.shanshu.ai/download/copt/4.0.7/win64/CardinalOptimizer-4.0.7-win64.tar.gz

[COPT-Public]

Interesting that, on Windows, the extracted *.exe files have "executable" access, but the *.dll files don't.

[lschork2]

We now use scripts/generate_tar_package.jl to convert the COPT zip package to tar.gz. When the script is used on Windows, then the .dll file obtained by extracting the tar.gz has executable permission and the automatic installation works. When the script is used on Linux, then the .dll file does not have executable permission and the automatic installation fails.

So for now we build the package on Windows.