set_silent does not work in a subproblem #495
Labels
Comments
What, exactly, is it printing? There is a known upstream bug in Gurobi 9.5. It will be fixed in their next release: jump-dev/Gurobi.jl#437 (comment) Aren't you missing a |
Gurobi output like this: Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 66636 rows, 92449 columns and 175613 nonzeros
Model fingerprint: 0xa56d1602
Coefficient statistics:
Matrix range [2e-03, 1e+02]
Objective range [4e-03, 1e+05]
Bounds range [6e-08, 1e+05]
RHS range [2e-02, 4e+03]
Concurrent LP optimizer: primal simplex, dual simplex, and barrier
Showing barrier log only...
Presolve removed 63755 rows and 82643 columns
Presolve time: 0.16s
Presolved: 2881 rows, 9806 columns, 15635 nonzeros
Ordering time: 0.00s
Barrier statistics:
AA' NZ : 5.821e+03
Factor NZ : 3.436e+04 (roughly 5 MB of memory)
Factor Ops : 4.919e+05 (less than 1 second per iteration)
Threads : 1
Objective Residual
Iter Primal Dual Primal Dual Compl Time
0 3.20694722e+09 -7.50426186e+08 5.87e+04 4.53e+03 1.11e+07 0s
1 1.43431565e+09 -1.10137235e+09 1.76e+04 1.40e+04 3.78e+06 0s
2 3.96512486e+08 -1.65832502e+09 3.36e+03 1.80e+03 8.01e+05 0s
3 8.28920489e+07 -1.07377175e+09 4.09e+02 2.11e+02 1.42e+05 0s
4 3.52591801e+07 -3.37530057e+08 7.11e+01 3.06e+01 3.09e+04 0s
5 1.97545688e+07 -1.52507306e+08 2.13e+01 1.11e+01 1.22e+04 0s
6 1.26250695e+07 -6.86791667e+07 1.03e+01 3.97e+00 5.44e+03 0s
7 7.90076843e+06 -2.65271395e+07 6.45e+00 1.84e+00 2.24e+03 0s
8 3.45599025e+06 -6.92775661e+06 2.93e+00 1.64e-01 6.59e+02 0s
9 1.78406418e+06 -3.08977091e+06 1.34e+00 6.30e-02 3.05e+02 0s
10 6.40889552e+05 -5.38370349e+05 2.99e-01 9.91e-03 7.31e+01 0s
11 2.86751694e+05 -6.83044898e+03 3.92e-02 2.10e-03 1.82e+01 0s
12 2.36546874e+05 9.70954514e+04 1.55e-02 7.32e-04 8.62e+00 0s
13 2.22895966e+05 1.28149085e+05 1.06e-02 4.58e-04 5.85e+00 0s
14 2.12243563e+05 1.51212307e+05 7.09e-03 2.61e-04 3.77e+00 0s
15 2.04354726e+05 1.63824500e+05 4.77e-03 1.37e-04 2.50e+00 0s
16 1.97942208e+05 1.74623421e+05 3.00e-03 5.17e-05 1.44e+00 0s
Barrier performed 16 iterations in 0.27 seconds (0.11 work units)
Barrier solve interrupted - model solved by another algorithm
Solved with dual simplex
Solved in 4322 iterations and 0.28 seconds (0.12 work units)
Optimal objective 1.849049734e+05
User-callback calls 148, time in user-callback 0.00 secI think for each node from the problem, because it prints this many times.
Yeah, i'm using set_silent(sp) in my subproblem definition.
This problem may be relationated with that bug? |
|
Try optimizer = optimizer_with_attributes(() -> Gurobi.Optimizer(gurobi_env), "OutputFlag" => 0),
) do sp, tor optimizer = () -> Gurobi.Optimizer(gurobi_env),
) do sp, t
set_optimizer_attribute(sp, "OutputFlag", 0) |
|
Actually i'm using set_optimizer_attribute(sp, "OutputFlag", 0)
set_silent(sp)This works with SDDP 0.4.3 and Gurobi 9.1.2, but with SDDP 0.4.4 and Gurobi 9.5 it doesnt. |
|
Please provide a reproducible example. |
|
Based on the hydrothermal scheduling examples: using Gurobi, SDDP
gurobi_env = Gurobi.Env()
graph = SDDP.LinearGraph(3)
SDDP.add_edge(graph, 3 => 1, 0.95)
model = SDDP.PolicyGraph(
graph,
sense = :Min,
lower_bound = 0.0,
optimizer = optimizer_with_attributes(() -> Gurobi.Optimizer(gurobi_env))) do sp, t
@variable(sp, 5 <= x <= 15, SDDP.State, initial_value = 10)
@variable(sp, g_t >= 0)
@variable(sp, g_h >= 0)
@variable(sp, s >= 0)
@constraint(sp, balance, x.out - x.in + g_h + s == 0)
@constraint(sp, demand, g_h + g_t == 0)
@stageobjective(sp, s + t * g_t)
SDDP.parameterize(sp, [[0, 7.5], [3, 5], [10, 2.5]]) do w
set_normalized_rhs(balance, w[1])
return set_normalized_rhs(demand, w[2])
end
set_optimizer_attribute(sp, "OutputFlag", 0)
set_silent(sp)
end
SDDP.train(model; iteration_limit = 100, print_level = 1) |
|
What is the output of: using Gurobi, SDDP
gurobi_env = Gurobi.Env()
model = SDDP.LinearPolicyGraph(
stages = 3,
lower_bound = 0.0,
optimizer = () -> Gurobi.Optimizer(gurobi_env),
) do sp, t
set_silent(sp)
@variable(sp, 5 <= x <= 15, SDDP.State, initial_value = 10)
@variable(sp, g_t >= 0)
@variable(sp, g_h >= 0)
@variable(sp, s >= 0)
@constraint(sp, balance, x.out - x.in + g_h + s == 0)
@constraint(sp, demand, g_h + g_t == 0)
@stageobjective(sp, s + t * g_t)
SDDP.parameterize(sp, [[0, 7.5], [3, 5], [10, 2.5]]) do w
set_normalized_rhs(balance, w[1])
set_normalized_rhs(demand, w[2])
end
end
SDDP.train(model; iteration_limit = 1, print_level = 1) |
|
With SDDP 0.4.4 and Gurobi 9.5, the output is this: Set parameter Username
Academic license - for non-commercial use only - expires 2022-03-22
------------------------------------------------------------------------------
SDDP.jl (c) Oscar Dowson, 2017-21
Problem
Nodes : 3
State variables : 1
Scenarios : 2.70000e+01
Existing cuts : false
Subproblem structure : (min, max)
Variables : (6, 6)
VariableRef in MOI.LessThan{Float64} : (1, 2)
VariableRef in MOI.GreaterThan{Float64} : (5, 5)
AffExpr in MOI.EqualTo{Float64} : (2, 2)
Options
Solver : serial mode
Risk measure : SDDP.Expectation()
Sampling scheme : SDDP.InSampleMonteCarlo
Numerical stability report
Non-zero Matrix range [1e+00, 1e+00]
Non-zero Objective range [1e+00, 3e+00]
Non-zero Bounds range [5e+00, 2e+01]
Non-zero RHS range [2e+00, 1e+01]
No problems detected
Iteration Simulation Bound Time (s) Proc. ID # Solves
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Model fingerprint: 0x08fab8fe
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 1e+00]
Bounds range [5e+00, 2e+01]
RHS range [8e+00, 8e+00]
Presolve removed 2 rows and 6 columns
Presolve time: 0.00s
Presolve: All rows and columns removed
Iteration Objective Primal Inf. Dual Inf. Time
0 2.5000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 0 iterations and 0.00 seconds (0.00 work units)
Optimal objective 2.500000000e+00
User-callback calls 35, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Model fingerprint: 0x4f9f7d38
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 2e+00]
Bounds range [5e+00, 2e+01]
RHS range [3e+00, 1e+01]
Presolve removed 2 rows and 6 columns
Presolve time: 0.00s
Presolve: All rows and columns removed
Iteration Objective Primal Inf. Dual Inf. Time
0 0.0000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 0 iterations and 0.00 seconds (0.00 work units)
Optimal objective 0.000000000e+00
User-callback calls 30, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Model fingerprint: 0xf966aa66
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 3e+00]
Bounds range [5e+00, 2e+01]
RHS range [3e+00, 5e+00]
Presolve removed 1 rows and 3 columns
Presolve time: 0.00s
Presolved: 1 rows, 3 columns, 3 nonzeros
Iteration Objective Primal Inf. Dual Inf. Time
0 -1.6500000e+01 2.750000e+00 0.000000e+00 0s
1 0.0000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 1 iterations and 0.00 seconds (0.00 work units)
Optimal objective 0.000000000e+00
User-callback calls 53, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 3e+00]
Bounds range [5e+00, 2e+01]
RHS range [8e+00, 8e+00]
Iteration Objective Primal Inf. Dual Inf. Time
0 0.0000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 0 iterations and 0.00 seconds (0.00 work units)
Optimal objective 0.000000000e+00
User-callback calls 69, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 3e+00]
Bounds range [5e+00, 2e+01]
RHS range [3e+00, 5e+00]
Iteration Objective Primal Inf. Dual Inf. Time
0 0.0000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 0 iterations and 0.00 seconds (0.00 work units)
Optimal objective 0.000000000e+00
User-callback calls 85, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 3e+00]
Bounds range [5e+00, 2e+01]
RHS range [3e+00, 1e+01]
Iteration Objective Primal Inf. Dual Inf. Time
0 0.0000000e+00 2.500000e+00 0.000000e+00 0s
1 5.0000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 1 iterations and 0.00 seconds (0.00 work units)
Optimal objective 5.000000000e+00
User-callback calls 104, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 3 rows, 6 columns and 8 nonzeros
Coefficient statistics:
Matrix range [3e-01, 1e+00]
Objective range [1e+00, 2e+00]
Bounds range [5e+00, 2e+01]
RHS range [2e+00, 8e+00]
Iteration Objective Primal Inf. Dual Inf. Time
0 0.0000000e+00 3.750000e+00 0.000000e+00 0s
1 1.5000000e+01 0.000000e+00 0.000000e+00 0s
Solved in 1 iterations and 0.00 seconds (0.00 work units)
Optimal objective 1.500000000e+01
User-callback calls 49, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 3 rows, 6 columns and 8 nonzeros
Coefficient statistics:
Matrix range [3e-01, 1e+00]
Objective range [1e+00, 2e+00]
Bounds range [5e+00, 2e+01]
RHS range [2e+00, 5e+00]
Iteration Objective Primal Inf. Dual Inf. Time
0 4.0000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 0 iterations and 0.00 seconds (0.00 work units)
Optimal objective 4.000000000e+00
User-callback calls 65, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 3 rows, 6 columns and 8 nonzeros
Coefficient statistics:
Matrix range [3e-01, 1e+00]
Objective range [1e+00, 2e+00]
Bounds range [5e+00, 2e+01]
RHS range [2e+00, 1e+01]
Iteration Objective Primal Inf. Dual Inf. Time
0 -1.5000000e+01 7.500000e+00 0.000000e+00 0s
2 1.6666667e+00 0.000000e+00 0.000000e+00 0s
Solved in 2 iterations and 0.00 seconds (0.00 work units)
Optimal objective 1.666666667e+00
User-callback calls 84, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 3 rows, 6 columns and 8 nonzeros
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 1e+00]
Bounds range [5e+00, 2e+01]
RHS range [8e+00, 1e+01]
Iteration Objective Primal Inf. Dual Inf. Time
0 2.5000000e+00 8.611111e-01 0.000000e+00 0s
2 8.2777778e+00 0.000000e+00 0.000000e+00 0s
Solved in 2 iterations and 0.00 seconds (0.00 work units)
Optimal objective 8.277777778e+00
User-callback calls 54, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 3 rows, 6 columns and 8 nonzeros
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 1e+00]
Bounds range [5e+00, 2e+01]
RHS range [3e+00, 1e+01]
Iteration Objective Primal Inf. Dual Inf. Time
0 2.1111111e+00 3.611111e-01 0.000000e+00 0s
1 2.6363636e+00 0.000000e+00 0.000000e+00 0s
Solved in 1 iterations and 0.00 seconds (0.00 work units)
Optimal objective 2.636363636e+00
User-callback calls 73, time in user-callback 0.00 sec
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
Optimize a model with 3 rows, 6 columns and 8 nonzeros
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [1e+00, 1e+00]
Bounds range [5e+00, 2e+01]
RHS range [3e+00, 1e+01]
Iteration Objective Primal Inf. Dual Inf. Time
0 -6.8636364e+00 6.863636e+00 0.000000e+00 0s
2 2.5000000e+00 0.000000e+00 0.000000e+00 0s
Solved in 2 iterations and 0.00 seconds (0.00 work units)
Optimal objective 2.500000000e+00
User-callback calls 92, time in user-callback 0.00 sec
1 2.500000e+00 4.471380e+00 1.010100e+01 1 12
Terminating training
Status : iteration_limit
Total time (s) : 1.010100e+01
Total solves : 12
Best bound : 4.471380e+00
Simulation CI : 2.500000e+00 ± NaN
------------------------------------------------------------------------------The same script, but now with SDDP 0.4.3 and Gurobi 9.1.2, gives this output: Academic license - for non-commercial use only - expires 2022-03-22
------------------------------------------------------------------------------
SDDP.jl (c) Oscar Dowson, 2017-21
Problem
Nodes : 3
State variables : 1
Scenarios : 2.70000e+01
Existing cuts : false
Subproblem structure : (min, max)
Variables : (6, 6)
VariableRef in MOI.LessThan{Float64} : (1, 2)
VariableRef in MOI.GreaterThan{Float64} : (5, 5)
AffExpr in MOI.EqualTo{Float64} : (2, 2)
Options
Solver : serial mode
Risk measure : SDDP.Expectation()
Sampling scheme : SDDP.InSampleMonteCarlo
Numerical stability report
Non-zero Matrix range [1e+00, 1e+00]
Non-zero Objective range [1e+00, 3e+00]
Non-zero Bounds range [5e+00, 2e+01]
Non-zero RHS range [2e+00, 1e+01]
No problems detected
Iteration Simulation Bound Time (s) Proc. ID # Solves
1 1.950000e+01 9.166667e+00 1.408000e+01 1 12
Terminating training
Status : iteration_limit
Total time (s) : 1.408000e+01
Total solves : 12
Best bound : 9.166667e+00
Simulation CI : 1.950000e+01 ± NaN
------------------------------------------------------------------------------ |
|
What is this "Username" parameter? Are you using a remote license? You might need using Gurobi, SDDP
gurobi_env = Gurobi.Env()
Gurobi.GRBsetintparam(gurobi_env, "OutputFlag", 0) |
I don't know. I'm using a personal academic license, alocated in my license.lic file.
This actually worked. Maybe with the new SDDP 0.4.4 version, the line Thanks! |
No, this isn't related to SDDP.jl or Gurobi.jl. It's likely the same upstream bug in Gurobi 9.5. Since there is a work-around, I'm going to close this issue. Either use Gurobi 9.1, or apply the work-around. |
|
Hi, I'm having the same issue with SDDP 0.4.4 and CPLEX 0.8.1 (details below), @odow could you please provide the equivalent work around for
with CPLEX ? using the same MWE : using SDDP, CPLEX
graph = SDDP.LinearGraph(3)
model = SDDP.PolicyGraph(
graph,
sense = :Min,
lower_bound = 0.0,
optimizer = CPLEX.Optimizer,
) do sp, t
set_silent(sp) # !!!
@variable(sp, 5 <= x <= 15, SDDP.State, initial_value = 10)
@variable(sp, g_t >= 0)
@variable(sp, g_h >= 0)
@variable(sp, s >= 0)
@constraint(sp, balance, x.out - x.in + g_h + s == 0)
@constraint(sp, demand, g_h + g_t == 0)
@stageobjective(sp, s + t * g_t)
SDDP.parameterize(sp, [[0, 7.5], [3, 5], [10, 2.5]]) do w
set_normalized_rhs(balance, w[1])
return set_normalized_rhs(demand, w[2])
end
end
SDDP.train(model, iteration_limit = 1)I get |
|
You probably need to set some additional CPLEX arguments? CPLEX.jl sets |
|
Sorry for my late reply, but I still can't get CPLEX silent with my set up described above. I tried both |
|
I don't have CPLEX installed to test unfortunately. Do you have a reproducible example? |
|
What about setting |
|
I'm off the lab right now so I'll test it tomorrow. Meanwhile, as a reproducible example I used the one of my first post, trying to change |
|
so I've tried several ways to turn CPLEX silent in a SDDP model, see the commented lines in the following MWE : using SDDP, CPLEX
graph = SDDP.LinearGraph(3)
model = SDDP.PolicyGraph(
graph,
sense = :Min,
lower_bound = 0.0,
optimizer = CPLEX.Optimizer,
) do sp, t
set_silent(sp)
#set_optimizer_attribute(sp, "CPXPARAM_ScreenOutput", 0)
#MOI.set(sp, MOI.RawOptimizerAttribute("CPXPARAM_ScreenOutput"), 0)
#MOI.set(sp, MOI.RawOptimizerAttribute("CPX_PARAM_SCRIND"), 0)
@variable(sp, 5 <= x <= 15, SDDP.State, initial_value = 10)
@variable(sp, g_t >= 0)
@variable(sp, g_h >= 0)
@variable(sp, s >= 0)
@constraint(sp, balance, x.out - x.in + g_h + s == 0)
@constraint(sp, demand, g_h + g_t == 0)
@stageobjective(sp, s + t * g_t)
SDDP.parameterize(sp, [[0, 7.5], [3, 5], [10, 2.5]]) do w
set_normalized_rhs(balance, w[1])
return set_normalized_rhs(demand, w[2])
end
end
SDDP.train(model, iteration_limit = 1, print_level=0)in all cases, I get undesired information printed (as in my initial post above). This is embarrassing since it slows down SDDP a lot for larger models. Yet with the same set up (CPLEX v0.8.1 + JuMP v0.22.2 + SDDP v0.4.4) the following simple JuMP model is silent : m = Model(CPLEX.Optimizer)
@variable(m, x >= 0)
@objective(m, Min, x)
set_silent(m)
optimize!(m)should I open a new issue for this ? |
|
Actually this seems not to be solver dependent: I have the same issue when replacing CPLEX with Clp in the MWE |
|
Ah. This is actually a MOI issue: using JuMP, Clp
model = Model()
@variable(model, x)
set_silent(model)
set_optimizer(model, Clp.Optimizer)
optimize!(model)As a temporary work-around, use: using SDDP, CPLEX
model = SDDP.LinearPolicyGraph(
stages = 3,
sense = :Min,
lower_bound = 0.0,
optimizer = optimizer_with_attributes(CPLEX.Optimizer, MOI.Silent() => true),
) do sp, t
@variable(sp, 5 <= x <= 15, SDDP.State, initial_value = 10)
@variable(sp, g_t >= 0)
@variable(sp, g_h >= 0)
@variable(sp, s >= 0)
@constraint(sp, balance, x.out - x.in + g_h + s == 0)
@constraint(sp, demand, g_h + g_t == 0)
@stageobjective(sp, s + t * g_t)
SDDP.parameterize(sp, [[0, 7.5], [3, 5], [10, 2.5]]) do w
set_normalized_rhs(balance, w[1])
set_normalized_rhs(demand, w[2])
return
end
end
SDDP.train(model, iteration_limit = 1, print_level=0) |
odow
changed the title
|
Closing because this was a bug in MathOptInterface. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have a problem using the most recent SDDP package version with Gurobi 9.5.
When i try tu run a model, it makes multiple prints in each interation, son the overall problem takes more time to solve due the large amount of time used printing each subproblem resolution.
I'm running the model the same way that in the past 0.4.3 SDDP version, with this line:
and i create the model this way:
Maybe Gurobi 9.5 changed something that force to print all the subproblem solves?
I'm using SDDP 0.4.3 with Gurobi 9.1.2 because of this, but i want to try the new version because it may make me get better solve times for the overall model.
Thanks in advance!
The text was updated successfully, but these errors were encountered: