Prototyping custom model/optimizer

[guimarqu]

First implementation for the custom model/optimizer.

At the moment it's not possible to replace the Formulation by a custom one because it requires much more work.

[guimarqu]

@laradicp I wasn't working when direct mode was deactivated, so I made some modifications.

However, I don't think it's a good solution because I think we should build the mapping custom model decision -> VarId once (at the beginning).

It's more difficult than I thought. We must figure out how we can build this relation custom model decision -> VarId knowing that the user can only give a relation between custom model decision and JuMP.VariableRef.

My first idea was to use MOI.VariableIndex -> VarId from Optimizer.varids but we don't have access to the optimizer in the run! method of the custom optimizer. Therefore, I tried to move Optimizer.varids into Env , but it breaks everything (circular dep Env & MathProg). (now it works because the env arg of create_formulation is not typed anymore.

[guimarqu]

Tests are ok

[rrsadykov]

I find this organization strange. You have subproblem formulation, inside you have optimizer, which itself has AbstractFormulation inside (I think that AbstractFormulation should be renamed to AbstractModel). It would be much simpler to directly have an AbstractModel as a subproblem formulation.

In specify! instead giving solver, we may define directly a model. For me it should be something like

mutable struct KnapsackLibModel <: Coluna.AbstractModel
    master::Coluna.MathProg.Formulation
    nbitems::Int
    costs::Vector{Float64}
    weights::Vector{Float64}
    capacity::Float64
    job_to_jumpvar::Dict{Int, JuMP.VariableRef}
end

function setmaster!(knap_model::KnapsackLibModel, master::Coluna.MathProg.Formulation)
    knap_model.master = master
end

struct KnapsackAlgorithm <: Coluna.AbstractOptimisationAlgorithm
end

function Coluna.Algorithm.run!(
    algo::KnapsackAlgorithm, env::Coluna.Env, model::KnapsackLibModel,
    input::Coluna.Algorithm.OptimizationInput)
    costs = -[Coluna.MathProg.getcurcost(model.master, _getvarid(model.master, env, job_to_jump_var[j])) for j in 1:length(opt.model.costs)]
    ...
    for j in selected
        push!(varids, _getvarid(model.master, env, job_to_jump_var[j]))
        push!(varvals, 1)
    end
    ...

end

....

coluna = JuMP.optimizer_with_attributes(
    Coluna.Optimizer,
    "params" => CL.Params(solver = TreeSearchAlgorithm(
        conqueralg=ColCutGenConquer(
            colgen=ColumnGeneration(pricing_prob_solve_alg=KnapsackAlgorithm())
        )
    )),
    "default_optimizer" => GLPK.Optimizer
)

model = BlockModel(coluna; direct_model = true)
@variable(model, x[m in data.machines, j in data.jobs], Bin)
@constraint(model, 
    sp[j in data.jobs], sum(x[m,j] for m in data.machines) == 1
)
@objective(model, Min,
    sum(data.cost[j,m]*x[m,j] for m in data.machines, j in data.jobs)
)

@axis(M, data.machines)
@dantzig_wolfe_decomposition(model, dec, M)

sp = getsubproblems(dec)
for m in M
    knp_model = KnapsackModel(length(data.jobs))
    setcapacity!(knp_model, data.capacity[m])
    for j in data.jobs
        setweight!(knp_model, j, data.weight[j,m])
        setcost!(knp_model, j, data.cost[j,m])
        map!(knp_model, j, x[m,j])
        #here you need to say that x[m,j] is an implicit representative variable
    end
    specify!(sp[m], model = knp_model) 
end

We may define AbstractCustomSpModel <: AbstractModel for which function setmaster! should be defined. So Coluna will call this function if it sees an AbstractCustomSpModel.

There is also a functionality missing, which allows us to say that a certain variable is implicit representative.

Concerning setup variables, I think they should should be managed automatically by Coluna

[rrsadykov]

Ok, may be sp. setup variables should also be managed by the user. So we may have something like

@variable(model, setup[m in data.machines], Int)
...
set_setup_var!(knp_model[m], setup[m])
...
specify!(sp[m], model = knp_model, setupvar = setup[m]) 

With the knowledge of setup variable, the user can check whether a certain solution has a negative reduced cost. Here setup vars may be defined automatically by Coluna, but set_setup_var!() function still should be called in the beginning.

[guimarqu]

Follow up in #537

[rrsadykov]

Ok for me. Why the tests do not pass?

[guimarqu]

Because we need to release a new version of BlockDecomposition