Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force recompilation when deps.jl changes #438

Closed
ExpandingMan opened this issue Nov 19, 2021 · 10 comments · Fixed by #439
Closed

Force recompilation when deps.jl changes #438

ExpandingMan opened this issue Nov 19, 2021 · 10 comments · Fixed by #439
Labels
bug Help wanted! Installation troubles User couldn't install Gurobi.jl

Comments

@ExpandingMan
Copy link
Contributor

It's really nice that this is now downloading an artifact, however it's really screwing up a case in which I have an older computer server running which is incompatible with 0.9.5.

I was hoping that by setting GUROBI_HOME when Gurobi.jl gets built it would use that and not try to use the artifact, however I couldn't seem to get it to do this.

I suggest

  • If no GUROBI_HOME is set, use artifacts as it currently does.
  • If GUROBI_HOME is set, skip artifacts and just use that.
@ExpandingMan ExpandingMan changed the title make it easier to use existing environment make it easier to use existing binaries Nov 19, 2021
@ExpandingMan
Copy link
Contributor Author

Looks like what I described above was intended... seems to be a very strange bug... _DEPS_FILE exists but it is using the artifact version of ligburobi... very weird...

@ExpandingMan
Copy link
Contributor Author

Figured it out... apparently Julia bakes the value of libgurobi into the pre-compile cache. For this package, this should probably be circumvented so that one can change the binary location just by doing Pkg.build("Gurobi"), though I'm not sure what is the most appropriate way of doing that.

@odow
Copy link
Member

odow commented Nov 20, 2021

We use existing binaries if a deps.jl file is present:

if isfile(_DEPS_FILE)
include(_DEPS_FILE)

so that one can change the binary location just by doing Pkg.build("Gurobi")

Does this not work? If re-building a package doesn't force recompilation, that should be a Julia problem. I don't think we can fix that in Gurobi.jl.

I don't have a linux machine to test this further, unfortunately.

@odow odow added the Installation troubles User couldn't install Gurobi.jl label Nov 20, 2021
@ExpandingMan
Copy link
Contributor Author

Like I said, I actually had to delete the pre-compile cache before it would use a new value for libgurobi. This seems like it warrants a fix, but I'm not sure what the best way of fixing it is. A solution which would certainly work but is rather hackish and seems liable to have undesirable side effects would be to just delete the pre-compile cache (I'm not sure off the top of my head if there is a Pkg API call for this).

I will ask around on zulip and slack, I'm sure someone will know what the "right" way of doing this is.

@odow
Copy link
Member

odow commented Nov 20, 2021

If re-building a package doesn't force recompilation, that should be a Julia problem

I'd regard this as a bug in Julia. Is this something we can actually fix in Gurobi.jl?

@ExpandingMan
Copy link
Contributor Author

ExpandingMan commented Nov 21, 2021

It's a bit more subtle than that, the *.ji files don't actually store compiled code. Furthermore I don't think const is intended to be used this way, it probably should be a Ref that gets selected when the package initializes.

Regardless, I see a few options here:

  • Do nothing, but document the issue so that users can delete the *.ji cache manually as needed.
  • Delete all gurobi *.ji files in the build script.
  • Use a Ref and check the presence of the library at package initialization.

I took a look here for how other libraries deal with this, and, at least in the Vulkan example, it uses a Ref, though as that package is an automated wrapper it's a less than ideal comparison. Other examples I thought of all use jll and work a bit differently.

@odow
Copy link
Member

odow commented Nov 21, 2021

Do you have a link to the slack conversation?

@ExpandingMan
Copy link
Contributor Author

We had this conversation on zulip. Most of what I said here was from my own investigation. Someone more familiar with Pkg related stuff might still have something valuable to contribute here.

@odow
Copy link
Member

odow commented Nov 21, 2021

We probably can make this work with an __init__ function. PRs accepted, but I have no plans to work on this.

The slack conversation (for the next few days before it gets purged) is: https://julialang.slack.com/archives/CBF3Z1D7V/p1637523109151900

@odow odow added the bug label Nov 21, 2021
@odow
Copy link
Member

odow commented Nov 21, 2021

@KristofferC 's suggestion from slack is to use https://docs.julialang.org/en/v1/base/base/#Base.include_dependency.

@odow odow changed the title make it easier to use existing binaries Force recompilation when deps.jl changes Nov 21, 2021
@odow odow closed this as completed in #439 Feb 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Help wanted! Installation troubles User couldn't install Gurobi.jl
Development

Successfully merging a pull request may close this issue.

2 participants