Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Numerical differences between versions 0.7.0 and 0.8.4 #120

Closed
s3baki opened this issue Sep 23, 2021 · 9 comments
Closed

Numerical differences between versions 0.7.0 and 0.8.4 #120

s3baki opened this issue Sep 23, 2021 · 9 comments

Comments

@s3baki
Copy link

s3baki commented Sep 23, 2021

When we moved from Clp 0.7.0 to Clp 0.8.4, I had to completely rewrite my interface to Clp.
Before I was using a very simple

using MathProgBase
using Clp
solution = linprog(obj, a, '=', rhs, lb, ub, ClpSolver(MaximumIterations = maxiter, MaximumSeconds = maxtime))

Now I use

using JuMP
using Clp
model = Model(Clp.Optimizer);
@variable(model, lb[i] <= x[i=1:length(lb)] <= ub[i]);
@objective(model, Min, obj'x);
# Depending on the `sense` vector, you may need to split it into `<=` and `>=` constraints too
@constraint(model, a * x .== rhs);
JuMP.set_optimizer_attribute(model, "LogLevel", 0)
JuMP.set_optimizer_attribute(model, "MaximumIterations", maxiter)
optimize!(model);

I don't mind the extra work of setting up the model.
I write that once and done.
What I do mind is that I am getting different results.
In most cases it is just numerical fuzz, but at times the differences are more noticeable.
Sometimes the objective value is better, and sometimes worse.

My questions are the following:

Has the underlying COIN code changed?

If not, are there some hidden settings that have changed, which causes the underlying C-library to calculate differently?

I am happy to provide examples of both numerical fuzz and substantial differences.
I can be reached at [email protected].

Any insight you can give me will be highly appreciated.
@odow
Copy link
Member

odow commented Sep 23, 2021

Has the underlying COIN code changed?

Yes.

Sometimes the objective value is better, and sometimes worse.

It seems like you're terminating the solver after a limited number of iterations. There is no guarantee that the solver will find identical solutions as a function of the iteration number.

If you have a situation where it finds different optimal solutions, that is worth investigating as a bug.

@s3baki
Copy link
Author

s3baki commented Oct 6, 2021 via email

@odow
Copy link
Member

odow commented Oct 6, 2021

Github strips attachments if you reply via email so I can't run it. You should upload manually using the web interface.

If you're finding different solutions due to a time out, this is expected. If you're finding different solutions with the same objective value, this is expected.

If you're finding optimal solutions with different objective values and you did not set a time limit or iteration limit, this is not expected.

@s3baki
Copy link
Author

s3baki commented Oct 7, 2021 via email

@odow
Copy link
Member

odow commented Oct 7, 2021

Please provide a reproducible example. You can't attach files via email, this must be done via the web interface.

@s3baki
Copy link
Author

s3baki commented Oct 8, 2021

[case_3_3.csv](https://github.com/jump-dev/Clp.jl/files/7309610/case_3_3.csv
case_5_8.csv
case_5_27.csv
)

@s3baki s3baki closed this as completed Oct 8, 2021
@odow odow reopened this Oct 8, 2021
@odow
Copy link
Member

odow commented Oct 8, 2021

I cannot reproduce this.

  • What is versioninfo()
  • What is ] st -m (type ] first to enter pkg mode, then the rest)
  • Please turn up the logging and provide logs of both cases
  • What are the two optimal objective values?
  • Please run without setting an iteration or time limit

Old

using MathProgBase # version v0.7.8
using Clp # version v0.7.0

using DelimitedFiles
indata = readdlm("case_3_3.csv", ',')
lengths = [findfirst(x -> !isa(x,Number), indata[i,2:end]) for i in 1:first(size(indata))]
rows = lengths[1] - 1
cols = lengths[3] - 1
@assert size(indata)[2] == 1 + rows*cols

obj = Vector{Float64}(indata[1, 2:rows+1])
a   = Matrix{Float64}(reshape(indata[2, 2:end], (cols, rows)))
rhs = Vector{Float64}(indata[3, 2:cols+1])
lb  = Vector{Float64}(indata[4, 2:rows+1])
ub  = Vector{Float64}(indata[5, 2:rows+1])
maxiter = indata[6, 2]
maxtime = indata[7, 2]

sol = linprog(obj, a, '=', rhs, lb, ub, ClpSolver(MaximumIterations = maxiter, MaximumSeconds = maxtime, SolveType = 0))
julia> sol.objval
28048.98063242901

(cl2) pkg> st
      Status `/private/tmp/cl2/Project.toml`
  [e2554f3b] Clp v0.7.0
  [fdba3010] MathProgBase v0.7.8

New

using JuMP # version 0.21.10
using Clp # version 0.8.4

function decision_solver(
    obj::Vector{Float64},
    a::Matrix{Float64},
    rhs::Vector{Float64},
    ub::Vector{Float64},
    lb::Vector{Float64},
    maxtime::Int64,
    maxiter::Int64)
    model = Model(Clp.Optimizer);
    @variable(model, lb[i] <= x[i=1:length(lb)] <= ub[i]);
    @objective(model, Min, obj'x);
    # Depending on the sense vector, you may need to split it into <= and >= constraints too
    @constraint(model, a * x .== rhs);
    # JuMP.set_optimizer_attribute(model, "LogLevel", 0)
    JuMP.set_optimizer_attribute(model, "MaximumIterations", maxiter)
    optimize!(model);
    status = Symbol(lowercase(string(termination_status(model))))
    @show status
    if status in [:infeasible, :dual_infeasible]
        JuMP.set_optimizer_attribute(model, "SolveType", 1)
        optimize!(model);
        status = Symbol(lowercase(string(termination_status(model))))
    end
    if status != :optimal
        return (status, 0.0, Vector{Float64}())
    end
    sol = value.(x)
    objval = objective_value(model)
    return (status, objval, sol)
end

using DelimitedFiles
indata = readdlm("case_3_3.csv", ',')
lengths = [findfirst(x -> !isa(x,Number), indata[i,2:end]) for i in 1:first(size(indata))]
rows = lengths[1] - 1
cols = lengths[3] - 1
@assert size(indata)[2] == 1 + rows*cols

obj = Vector{Float64}(indata[1, 2:rows+1])
a   = Matrix{Float64}(reshape(indata[2, 2:end], (cols, rows)))
rhs = Vector{Float64}(indata[3, 2:cols+1])
lb  = Vector{Float64}(indata[4, 2:rows+1])
ub  = Vector{Float64}(indata[5, 2:rows+1])
maxiter = indata[6, 2]
maxtime = indata[7, 2]

(status, objval, sol) = decision_solver(obj, a, rhs, ub, lb, Int64(maxtime), Int64(maxiter))

julia> objval
28048.98063242901

julia> objval
28048.98063242901

(cl) pkg> st
      Status `/private/tmp/cl/Project.toml`
  [e2554f3b] Clp v0.8.4
  [4076af6c] JuMP v0.21.10

@odow
Copy link
Member

odow commented Nov 4, 2021

@s3baki any follow up to this?

@odow
Copy link
Member

odow commented Nov 22, 2021

Closing as stale and unreproducible. @s3baki if you can reliably reproduce this on the latest version, please re-open.

@odow odow closed this as completed Nov 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@odow @s3baki