Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to release a Gurobi license? #268

Closed
blimasouza opened this issue Nov 8, 2019 · 11 comments
Closed

How to release a Gurobi license? #268

blimasouza opened this issue Nov 8, 2019 · 11 comments

Comments

@blimasouza
Copy link

When using a Gurobi token server, I often need to make sure to release my license after using it. With python I would run:

del model
gc.collect()
disposeDefaultEnv()

How would I achieve the same using JuMP?

@blegat blegat transferred this issue from jump-dev/JuMP.jl Nov 8, 2019
@odow
Copy link
Member

odow commented Nov 8, 2019

You want something like:

using JuMP, Gurobi
env = Gurobi.Env()
model = Model(with_optimizer(Gurobi.Optimizer, env))
# ...
Gurobi.free_env(env)

@odow
Copy link
Member

odow commented Nov 8, 2019

@blimasouza
Copy link
Author

blimasouza commented Nov 8, 2019

Cool! Thanks for the quick answer! Your solution works perfectly well. The license is being correctly released.
Sometimes I get the following warning message:

Warning: invalid Gurobi environment. Was it freed too early?

Would you know why?

The piece of code that I am playing with is:

using JuMP, Gurobi, Test
const MOI = JuMP.MathOptInterface

"""
    example_knapsack(; verbose = true)
Formulate and solve a simple knapsack problem:
    max sum(p_j x_j)
     st sum(w_j x_j) <= C
        x binary
"""
function example_knapsack(; verbose=true)
    profit = [5, 3, 2, 7, 4]
    weight = [2, 8, 4, 2, 5]
    capacity = 10
    env = Gurobi.Env()
    model = Model(with_optimizer(Gurobi.Optimizer, env))
    @variable(model, x[1:5], Bin)
    # Objective: maximize profit
    @objective(model, Max, profit' * x)
    # Constraint: can carry all
    @constraint(model, weight' * x <= capacity)
    # Solve problem using MIP solver
    JuMP.optimize!(model)
    if verbose
        println("Objective is: ", JuMP.objective_value(model))
        println("Solution is:")
        for i in 1:5
            print("x[$i] = ", JuMP.value(x[i]))
            println(", p[$i]/w[$i] = ", profit[i] / weight[i])
        end
    end
    @test JuMP.termination_status(model) == MOI.OPTIMAL
    @test JuMP.primal_status(model) == MOI.FEASIBLE_POINT
    @test JuMP.objective_value(model) == 16.0
    
    Gurobi.free_env(env)
end

Then, if I evaluate a few times the function example_knapsack, I get:

for i in 1:3
    println("Round $i")
    println("-------")
    example_knapsack(verbose=false)
    println("-------")
end
Round 1
-------
Warning: invalid Gurobi environment. Was it freed too early?
Warning: invalid Gurobi environment. Was it freed too early?
Warning: invalid Gurobi environment. Was it freed too early?
Warning: invalid Gurobi environment. Was it freed too early?
Optimize a model with 1 rows, 5 columns and 5 nonzeros
Variable types: 0 continuous, 5 integer (5 binary)
Coefficient statistics:
  Matrix range     [2e+00, 8e+00]
  Objective range  [2e+00, 7e+00]
  Bounds range     [0e+00, 0e+00]
  RHS range        [1e+01, 1e+01]
Found heuristic solution: objective 8.0000000
Presolve removed 1 rows and 5 columns
Presolve time: 0.00s
Presolve: All rows and columns removed

Explored 0 nodes (0 simplex iterations) in 0.00 seconds
Thread count was 1 (of 8 available processors)

Solution count 2: 16 8 

Optimal solution found (tolerance 1.00e-04)
Best objective 1.600000000000e+01, best bound 1.600000000000e+01, gap 0.0000%
-------
Round 2
-------
Optimize a model with 1 rows, 5 columns and 5 nonzeros
Variable types: 0 continuous, 5 integer (5 binary)
Coefficient statistics:
  Matrix range     [2e+00, 8e+00]
  Objective range  [2e+00, 7e+00]
  Bounds range     [0e+00, 0e+00]
  RHS range        [1e+01, 1e+01]
Found heuristic solution: objective 8.0000000
Presolve removed 1 rows and 5 columns
Presolve time: 0.00s
Presolve: All rows and columns removed

Explored 0 nodes (0 simplex iterations) in 0.00 seconds
Thread count was 1 (of 8 available processors)

Solution count 2: 16 8 

Optimal solution found (tolerance 1.00e-04)
Best objective 1.600000000000e+01, best bound 1.600000000000e+01, gap 0.0000%
-------
Round 3
-------
Optimize a model with 1 rows, 5 columns and 5 nonzeros
Variable types: 0 continuous, 5 integer (5 binary)
Coefficient statistics:
  Matrix range     [2e+00, 8e+00]
  Objective range  [2e+00, 7e+00]
  Bounds range     [0e+00, 0e+00]
  RHS range        [1e+01, 1e+01]
Found heuristic solution: objective 8.0000000
Presolve removed 1 rows and 5 columns
Presolve time: 0.00s
Presolve: All rows and columns removed

Explored 0 nodes (0 simplex iterations) in 0.00 seconds
Thread count was 1 (of 8 available processors)

Solution count 2: 16 8 

Optimal solution found (tolerance 1.00e-04)
Best objective 1.600000000000e+01, best bound 1.600000000000e+01, gap 0.0000%
-------

BTW: I am using Julia 1.1.1, JuMP v0.19.1 and Gurobi v0.6.0

@odow
Copy link
Member

odow commented Nov 8, 2019

I usually make env a global constant so that I only need one Gurobi license in a Julia session.

using JuMP, Gurobi, Test
const MOI = JuMP.MathOptInterface
const env = Gurobi.Env()

"""
    example_knapsack(; verbose = true)
Formulate and solve a simple knapsack problem:
    max sum(p_j x_j)
     st sum(w_j x_j) <= C
        x binary
"""
function example_knapsack(; verbose=true)
    profit = [5, 3, 2, 7, 4]
    weight = [2, 8, 4, 2, 5]
    capacity = 10
    model = Model(with_optimizer(Gurobi.Optimizer, env))
    @variable(model, x[1:5], Bin)
    # Objective: maximize profit
    @objective(model, Max, profit' * x)
    # Constraint: can carry all
    @constraint(model, weight' * x <= capacity)
    # Solve problem using MIP solver
    JuMP.optimize!(model)
    if verbose
        println("Objective is: ", JuMP.objective_value(model))
        println("Solution is:")
        for i in 1:5
            print("x[$i] = ", JuMP.value(x[i]))
            println(", p[$i]/w[$i] = ", profit[i] / weight[i])
        end
    end
    @test JuMP.termination_status(model) == MOI.OPTIMAL
    @test JuMP.primal_status(model) == MOI.FEASIBLE_POINT
    @test JuMP.objective_value(model) == 16.0
end

The warnings are probably coming from Julia calling free_env, and then trying to gc the model, only to find that the environment has already been freed.

@danielolsen
Copy link

I am having a similar problem releasing a Gurobi license after a model has been optimized.

The below code:

import Gurobi
import JuMP
env = Gurobi.Env()
m = JuMP.Model(JuMP.with_optimizer(Gurobi.Optimizer, env))
JuMP.@variable(m, x >= 0)
JuMP.@objective(m, Min, x)
JuMP.optimize!(m)
Gurobi.free_env(env)

produces the following error trace:

Compute Server communication statistics:
  Sent: 0.0 MBytes in 29 msgs and 0.00s (0.00 MB/s)
  Received: 0.0 MBytes in 21 msgs and 0.00s (0.00 MB/s)


Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffad27effa9 -- GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
in expression starting at REPL[9]:1
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
macro expansion at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_common.jl:60 [inlined]
free_env at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_env.jl:32
unknown function (ip: 0000000028A3F1B9)
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
do_call at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:328
eval_value at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:417
eval_stmt_value at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:368 [inlined]
eval_body at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:764
jl_interpret_toplevel_thunk_callback at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:888
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 0000000033BC888F)
unknown function (ip: 0000000000000000)
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:814
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:764
jl_toplevel_eval at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:823 [inlined]
jl_toplevel_eval_in at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:843
eval at .\boot.jl:330
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2135 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
eval_user_input at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:86
macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:118 [inlined]
#26 at .\task.jl:333
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2135 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
start_task at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\task.c:659
Allocations: 84707750 (Pool: 84697376; Big: 10374); GC: 73

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffad27effa9 -- GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
in expression starting at REPL[9]:1
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
macro expansion at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_common.jl:60 [inlined]
free_model at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_model.jl:83 [inlined]
#4 at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_model.jl:20
unknown function (ip: 0000000028A40049)
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
run_finalizer at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gc.c:231
jl_gc_run_finalizers_in_list at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gc.c:317
jl_gc_run_all_finalizers at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gc.c:345
jl_atexit_hook at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\init.c:257
jl_exit at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\jl_uv.c:629
jl_exception_handler at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\signals-win.c:308
__julia_personality at /cygdrive/d/buildbot/worker/package_win64/build/src/support/cygdrive/d/buildbot/worker/package_win64/build/src/support\win32_ucontext.c:28
_chkstk at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RtlRaiseException at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
KiUserExceptionDispatcher at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
GRBXaddrangeconstrs at C:\gurobi800\win64\bin\gurobi80.DLL (unknown line)
macro expansion at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_common.jl:60 [inlined]
free_env at C:\Users\dolsen\.julia\packages\Gurobi\CI8ht\src\grb_env.jl:32
unknown function (ip: 0000000028A3F1B9)
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
do_call at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:328
eval_value at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:417
eval_stmt_value at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:368 [inlined]
eval_body at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:764
jl_interpret_toplevel_thunk_callback at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:888
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 0000000033BC888F)
unknown function (ip: 0000000000000000)
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:814
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:764
jl_toplevel_eval at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:823 [inlined]
jl_toplevel_eval_in at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:843
eval at .\boot.jl:330
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2135 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
eval_user_input at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:86
macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:118 [inlined]
#26 at .\task.jl:333
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2135 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
start_task at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\task.c:659
Allocations: 84707750 (Pool: 84697376; Big: 10374); GC: 73

@odow
Copy link
Member

odow commented Jan 23, 2020

So the issue is calling free_env while the model is still in scope.

You really need something like

import Gurobi
import JuMP

function solve_problem(env)
    m = JuMP.Model(JuMP.with_optimizer(Gurobi.Optimizer, env))
    JuMP.@variable(m, x >= 0)
    JuMP.@objective(m, Min, x)
    JuMP.optimize!(m)
end

function solve_and_cleanup()
    env = Gurobi.Env()
    solve_problem(env)
    GC.gc()
    Gurobi.free_env(env)
end

solve_and_cleanup()

@danielolsen
Copy link

Thanks, that solves the problem! So if I understand this correctly, once the model and the optimizer become associated, there's no way to free the Gurobi env while the model is still in scope?

Just for reference, I get the same error when I first define the Model without an optimizer and then create one during the optimize! call, e.g.:

import Gurobi
import JuMP

env = Gurobi.Env()
m = JuMP.Model()
JuMP.@variable(m, x >= 0)
JuMP.@objective(m, Min, x)
JuMP.optimize!(m, JuMP.with_optimizer(Gurobi.Optimizer, env))
Gurobi.free_env(env)

@odow
Copy link
Member

odow commented Jan 23, 2020

So if I understand this correctly, once the model and the optimizer become associated, there's no way to free the Gurobi env while the model is still in scope?

There is one subtlety. It's the Gurobi model in scope, not the JuMP model. The Gurobi model is the underlying C object which users should never really interact with.

You could try:

using JuMP, Gurobi
model = Model()
env = Gurobi.Env()
set_optimizer(model, with_optimizer(Gurobi.Optimizer, env))
optimize!(model)
MOIU.drop_optimizer(model)
GC.gc()
Gurobi.free_env(env)

A better question is why you want to repeatedly detach and attach a Gurobi.Optimizer object, freeing the env between solves, and without destroying the JuMP model.

@danielolsen
Copy link

MOIU.drop_optimizer() does the trick, thanks!

A better question is why you want to repeatedly detach and attach a Gurobi.Optimizer object, freeing the env between solves, and without destroying the JuMP model.

Mostly a lack of familiarity with the JuMP ecosystem, to be honest. Using Pyomo, once a model is solved all of the variable values are automatically loaded into the Pyomo model and the Gurobi model is no longer needed, so it auto-terminates. This comes in handy in situations where there's a cost for keeping a Gurobi model alive, like when using their cloud. But now that I play around with drop_optimizer and then querying the model, I realize that this is not the case in JuMP, which in retrospect makes sense with the Julia/JuMP goals of efficiency.

@odow
Copy link
Member

odow commented Jan 23, 2020

This comes in handy in situations where there's a cost for keeping a Gurobi model alive, like when using their cloud.

Ah that makes sense then. In this case we would recommend that you build the JuMP model, solve, grab all you need and then discard, calling GC.gc().

I realize that this is not the case in JuMP

Yes, once you call drop_optimizer, the solution values are unaccessible. That's why I recommend the solve_and_cleanup approach over drop_optimizer, because at that point there isn't much to be gained from having the JuMP model with no solutions.

Mostly a lack of familiarity with the JuMP ecosystem, to be honest. Using Pyomo

We take a considerably different approach to Pyomo. The closest Pyomo has to JuMP is their "persistent interfaces."

@odow
Copy link
Member

odow commented Sep 28, 2020

See my answer to #110 (comment)

using JuMP, Gurobi
env = Gurobi.Env()
model = Model(() -> Gurobi.Optimizer(env))
set_silent(model)
@variable(model, x)
@objective(model, Min, x^2)
optimize!(model)
# To ensure `env` is finalized, you must also finalize any models using env.
# If you use `Gurobi.Optimizer`, the order of finalizing `.model` and `env` 
# doesn't matter. If you use the C API directly, `.model` _MUST_ be 
# finalized first.
finalize(backend(model).optimizer.model)
finalize(env)

Or if in direct-mode:

using JuMP, Gurobi
env = Gurobi.Env()
model = direct_model(Gurobi.Optimizer(env))
set_silent(model)
@variable(model, x)
@objective(model, Min, x^2)
optimize!(model)
# To ensure `env` is finalized, you must also finalize any models using env.
# If you use `Gurobi.Optimizer`, the order of finalizing `.model` and `env` 
# doesn't matter. If you use the C API directly, `.model` _MUST_ be 
# finalized first.
finalize(backend(model))
finalize(env)

Note that you should only call finalize if you understand what this means. Any future calls involving model or env may result in crashes or undefined behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants