Names for Data/Storage/StorageState

[guimarqu]

I don't understand how you can restore the state of an empty storage ? The state should be empty ? Maybe we should rename AbstractStorageState to AbstractDataState because it's the state of the AbstractData ?

Originally posted by @guimarqu in #323

waiting to get some ideas.

---

EDIT (2021/03/01), 2nd EDIT (2021/03/18):

Todo:

• StorageState becomes Record
• Storage becomes (Storage)Unit
• change order of arguments in restore_from_record!
• StorageContainer becomes StorageUnitWrapper
• move storage into ColunaBase
• move StorageDict into Model, then delete AbstractData (new method in model interface)
• StorageDict becomes Storage
• unit tests on storage "public" methods

Decisions to make :

• method to instanciate records ? rename StorageState -> RecordState #470 (comment)
• method to instanciate units ?
• remove NOT_USED (because NOT_USED and READ_ONLY are the same) (@rrsadykov)

Improve dev Exp :

• RecordsVector -> Vector{Record}
• StorageDict ? -> Storage
• UnitTypePair = Pair{DataType, DataType} ? -> removed because now bijection between UnitType & RecordType
• UnitsUsageDict = Dict{Tuple{AbstractModel, UnitTypePair}, UnitAccessMode} ? -> UnitsUsage

Conversations to resolve :

• Rename Storage -> (Storage) Unit #471 (comment)
• Rename Record State -> Record #472 (comment)
• Bijection StorageUnit -> Record #518 (comment)

Variables to rename :

• storage / storage unit wrapper in storages.jl
• storagedict / storage

Needs documentation (or remove)

• initialize_storage_units
• collect_units_to_restore! (why a UnitsUsageDict as first arg ?)
• store_records(reform)
• store_records(reform, record_vector)
• getstorageunit(model, storage_unit_type)
• record_type

[rrsadykov]

We can rename EmptyStorage to FormulationStorage.
Renaming AbstractStorageState to AbstractDataState is ok for me.
I would also be good to rename RuntimeData (ColGenRuntimeData, TreeSearchRuntimeData, etc) to something else, as now there are two datas in the the algorithms (runtime data and reformulation data). This is not good for the code readability.

[guimarqu]

It would be nice if we could remove RunTimeData structures in the long run.

[rrsadykov]

We need this kind of structure to keep the intermediate data which is needed only during the run of the algorithm

[guimarqu]

If you look at column generation, the structure is useless. We use it because we didn't think carefuly about the code design.

[rrsadykov]

I agree that in column generation is not that useful. However, for example, in the tree search algorithm we need to store the tree somewhere.

[guimarqu]

Todo (cc @rrsadykov) :

Edit : move the list of tasks to do in top comment.

[guimarqu]

Review

• Coluna.jl/src/Algorithm/formstorages.jl

  Line 12 in d6ef181

  function apply_data!(form::Formulation, var::Variable, var_state::VarState)

  (not in the interface)

Public

Coluna.jl/src/Algorithm/storage.jl

Lines 329 to 359 in d6ef181

	function restore_from_records!(records::RecordsVector, units_to_restore::UnitsUsageDict)
	TO.@timeit Coluna._to "Restore/remove records" begin
	for (storagecont, recordid) in records
	mode = get(
	units_to_restore,
	(getmodel(storagecont), gettypepair(storagecont)),
	READ_ONLY
	)
	restore_from_record!(storagecont, recordid, mode)
	end
	end
	empty!(records) # vector of records should be emptied
	end
	function remove_records!(records::RecordsVector)
	TO.@timeit Coluna._to "Restore/remove records" begin
	for (storagecont, recordid) in records
	restore_from_record!(storagecont, recordid, NOT_USED)
	end
	end
	empty!(records) # vector of records should be emptied
	end
	function copy_records(records::RecordsVector)::RecordsVector
	recordscopy = RecordsVector()
	for (storagecont, recordid) in records
	push!(recordscopy, storagecont => recordid)
	increaseparticipation!(storagecont, recordid)
	end
	return recordscopy
	end

Coluna.jl/src/Algorithm/storage.jl

Lines 22 to 28 in d6ef181

	To store all storage units of a data, we use functions
	"store_records!(::AbstractData)::RecordsVector" or
	"copy_records(::RecordsVector)::RecordsVector"
	Every stored record should be removed or restored using functions
	"restore_from_records!(::RecordsVector,::UnitsUsageDict)"
	and "remove_records!(::RecordsVector)"

When you create a record vector with copy or store, you then must destruct records using restore or remove. Otherwise memory leaks.

Dev

• Create a unit storage <:AbstractUnitStorage
• Define 0 to n record(s) for this unit storage
• Define Pair : Type{<:Unit} => Type{<:Record} (if no record : use EmptyRecord)
• Define constructor for the abstract unit storage :
  

  Coluna.jl/src/Algorithm/storage.jl

  Lines 41 to 44 in d6ef181

  	For every unit a constructor should be defined which
  	takes a model as a parameter. This constructor is
  	called when the formulation is completely known so the data
  	can be safely computed.

• Define constructor for the record :
  

  Coluna.jl/src/Algorithm/storage.jl

  Lines 54 to 56 in d6ef181

  	For each record, a constructor should be defined which
  	takes a model and a unit as parameters. This constructor
  	is called during storing a unit.

• Define
  

  Coluna.jl/src/Algorithm/storage.jl

  Lines 60 to 66 in d6ef181

  	"""
  	restore_from_record!(model, unit, record_state)
  	This method should be defined for every triple (model type, unit type, record type)
  	used by an algorithm.
  	"""
  	restore_from_record!(model::AbstractModel, unit::AbstractStorageUnit, state::AbstractRecord) =