Implement data structure for (boolean) control

[SnippenE]

Part of #13

[SouthEndMusic]

Here is a concept for how to implement control (also relevant for #263), feel free to criticize.

Relevant node categories

There are 3 categories of node that are relevant for control:

• The nodes with state (e.g. Basin), whose state forms the input of the control logic;
• A newly introduced node type called Control, which implements the control logic;
• The nodes which are to be controlled, say TabulatedRatingCurve or Pump.

Tables

We introduce a new table called control for each node type that can be controlled. This table is similar to the time table currently available for TabulatedRatingCurve, but instead of the column time it has the column control_state, containing a descriptive identifier string for each control state.

The Control node has these associated tables:

• conditions, with columns:
  • node_id (int),
  • listen_node_id (int),
  • variable (str),
  • greater_than (float).
    This table defines conditions like 'the variable in the node with ID listen_node_id is greater than greater_than'.
• logic_mapping, with columns:
  • node_id (int),
  • affect_node_id (int),
  • true_conditions (not sure),
  • false_conditions (not sure),
  • control_state (str).
    This table states that the node with the given node ID is in the given control state if the given conditions are true and false respectively.
• minimal_time, with columns:
  • node_id (int),
  • control_state (str)
  • minimal_time (float).
    This table states for each control state what the minimal time is that a node spends in this control state, to avoid oscillation between control states.

Implementation in Julia core

Say we implement a Control struct, with the following fields:

• minimal_time, conditions, logic_mapping: see above.
• condition_value: a boolean vector stating the truth value of each condition at the current time.
• control_state: a dictionary from node ID to a tuple '(current control state, start time of current control state)'.

We add a control field of type Control to the Parameters struct. We update the control states using a ContinuousCallback (see in the docs here). The condition value is updated in the condition! function:

function condition!(u,t,integrator)
   p = integrator.p

   out = 1.0
   condition_index = 0

   for node_id, var, greater_than in p.control.conditions
      condition_index += 1

      # get value of variable
      value = get_value(node_id,var)
      diff = value - greater_than
      out *= diff
      p.control.condition_value[condition_index] = (diff > 0)
   end

   return out
end

The condition function essentially implements a polynomial function with a zero at each condition change. When a root of the condition function is found, the affect! function is called:

function affect!(integrator)
   p = integrator.p

   for node_id in p.control.control_state.keys()
      # - Check what the current control state for this node ID should be based on p.control.condition_value and p.control.logic_mapping
      # - Check whether this is different from the current control state for this node ID and whether the minimal time for the current control state has passed based on p.control.control_state[node_id]
      # - Adapt parameters accordingly
      ...

   end
end

Final remarks

• I'm not sure it is good practice to make condition a mutating function.
• I only mentioned 'greater than' conditions. This is because equality conditions are probably not useful and 'smaller than' conditions are essentially the negation of 'greater than' conditions.
• A condition on flow in stead of basin state can also be used, then we require the variable 'flow', and 'node_id' must be the 'out_id'.
• A way must be implemented to let a control state change after the minimal time if appropriate.
• In this concept it is not necessary to explicitly define edges for the control part of the graph, these edges can be inferred. It would be useful to do this, at least for plotting.

[visr]

Nice proposal! I think this approach is worth pursuing, it seems quite flexible. Some comments:

• greater_than instead of bigger_than sounds more mathy, and dt instead of Dt since we only use lowercase column names (or perhaps duration or period is better`)
• I wouldn't worry about condition mutating, it's not mutating anything that affects the calculation directly
• In the control tables, we also need the control node ID, since all control nodes will be there. This column is commonly called node_id. So perhaps we need listen_node_id in conditions and affect_node_id in logic_mapping or something like that.

A way must be implemented to let a control state change after the minimal time if appropriate.

Perhaps when we set the control state we can stop after dt when a resetter function resets the state. This part is probably a bit difficult, and perhaps we should just leave this whole minimum time part out in the initial implementation.

In this concept it is not necessary to explicitly define edges for the control part of the graph

This seems like quite a deviation from the rest. Do I understand correctly that essentially with this proposal there is no reason to ever have more than 1 control node?

[evetion]

Thanks for the write up!

just leave this whole minimum time part out in the initial implementation.

Agreed, this was never discussed and makes the implementation more complex.

A condition on flow in stead of basin state can also be used, then we require the variable 'flow', and 'node_id' must be the 'out_id'.

It's hard to discern what the node_ids are (whether source, target or the control itself) in your proposal.

---

Partial proposal:

We do use the Edge table for Control nodes, where from_node_id is the Control, and the to_node_id is the affected/controlled Node. This fits with existing code and plotting, indicating that the to_node_id is "controlled".

A Control / target is mandatory for all Control nodes, having a node_id, a target variable to control and a remark to describe the why of the control. Essentially, a Control can thus only control 1 variable of a target and I'd probably argue that there also can only be a single control for any target variable.

Now there's two things left, when to control (change) and to what value. While I'm confident about the above, this is more speculation.

For the change logic, I'd propose rules which target a certain state. By default this state is say 1, and at each callback, you check all the rules relating to the possible target states, with the minimum state taking precedence if multiple rules match. For example you could have Control / time with the fields:

• node_id (int)
• condition (AND / OR)
• timestamp (datetime/float) # later than this triggers
• state (int)

And another table Control / lookup

• node_id (int)
• condition (AND / OR)
• from_node_id (int)
• from_variable (string)
• larger_than (float) # in the future moving to a separate operator and value column (indicating ==, != than value)
• state (int)

So for each node, you check the rules for each unique state and combine them using the condition. You can thus combine :level > 100 and/or time later than timestamp. The condition can be left out in the first prototype, only allowing for a single rule for each state.

Now, what value to set with a new state? I'd argue that we don't save these values in the Control tables at all, but add state columns to all existing node types, so we can switch there. This also keeps the types of the columns sane, otherwise we end up with a Any column in a control table.

The only downside is that this is very discrete. Although that's exactly what's proposed, setting a pump to a percentage of some Q somewhere else is not (pragmatically) possible. Maybe we should name this DiscreteControl, with the other a LinearControl?

[gijsber]

Looks very interesting and probably sufficient universal for expanding.
In terms of naming I like listen_node_id and target_node_id