Feat/ocm/random params (#2231)

* • component.py
  docstring mod to **size**

* • optimizationcontrolmechanism.py:
  added feature_input_ports attribute and num_feature_input_ports property

* • optimizationcontrolmechanism.py:
  added feature_input_ports attribute and num_feature_input_ports property

• parameterestimationcomposition.py: fixed misplacement of its Parameters() attribute

* • optimizationcontrolmechanism.py:
  added feature_input_ports attribute and num_feature_input_ports property

• parameterestimationcomposition.py: fixed misplacement of its Parameters() attribute

• optimizationfunctions.py: made num_estimates a Parameter

* - modified test_mode_based_num_estimates

* -

* -

* • optimizationcontrolmechanism.py:
  - _instantiate_control_signals: random_seeds -> random_seed_mod_values

* • composition.py
  - _add_controller:  modifying to instantiate feature_input_ports if none are specified

* • composition.py:
  - add_controller:  now adds feature_input_ports for Compostion INPUT nodes if not state_features not specified

* -

* • composition.py
  - _add_controller:  modifying to instantiate feature_input_ports if none are specified

* • composition.py:
  - add_controller:  assign simulation_input_ports

* -

* • optimizationcontrolmechanism.py:
  - feature_input_ports -> state_input_ports
  - _instantiate_input_ports():
      state_features only allowed to specifying state_input_ports if
      agent_rep is a CompositionFunctionApproximator (i.e., model-free optimization)

• composition.py:
  - add_controller: adds state_input_ports to shadow INPUT Nodes of Composition
    if controller.agent_rep is Composition (model-based optimziation)
    or state_features have not been specified (for model-free optimizaton)

* -

* • optimizationcontrolmechanism.py:
  _instantiate_input_ports:
     reinstate allowance of state_features specification if agent_rep is a Composition
     (i.e., model-based optimization) as long as they are all INPUT Nodes of agent_rep

* -

* -

* • optimizationcontrolmechanism.py
  - _gen_llvm_evaluate_function: num_estimates -> num_estimates_per_trial

* -

* • optimizationcontrolmechanism.py
  - _gen_llvm_evaluate_function: num_estimates -> num_trial_per_estimate

* • optimizationcontrolmechanism.py
  - _gen_llvm_evaluate_function: num_estimates -> num_trials_per_estimate

* -

* -

* -

* -

* • composition.py
  - __init__: moved controller instantiation until after nodes, projections and pathways

* • composition.py
  - __init__: restored add_controller position

* llvm/struct generation: Make sure num_estimats per trial is always integer

Signed-off-by: Jan Vesely <[email protected]>

* -

* • composition.py:
  - _update_controller: added
  - add_controller and _analyze_graph(): call _update_controller

* -

* • composition.py
  _update_controller:  fixed to loop through all input_ports of comp INPUT nodes

* • test_control.py
  - test_agent_rep_assignement_as_controller_and_replacement:
      updated to test that shadowing projections to state_input_ports
      are properly added and deleted

* • optimizationfunctions.py:
  - _function: refactored to put use aggregation_function at end
  - _grid_evaluate:  still needs to return all_samples

* -

* • composition.py
  - added call to _update_controller to add_node
  - moved test for projections to controller.state_input_ports to run()

* -

* • composition.py:
  moved calls to _update_controller to _complete_init_of_partially_initialized_nodes
  moved _update_controller to ocm._update_state_input_ports

• optimizationcontrolmechanism.py:
  added _update_state_input_ports [**still needed work**]

* -

* • composition.py:
  moved calls to _update_controller to _complete_init_of_partially_initialized_nodes
  moved _update_controller to ocm._update_state_input_ports
  _instantiate_controller_shadow_projections [still needs to be implemented]

• optimizationcontrolmechanism.py:
  added _update_state_input_ports [**still needed work**]

* • composition.py
  added needs_update_controller

* -

* • composition.py:
  - implemented self.needs_update_controller
  - moved implementation of controlsignal projections
    from add_controller to _instantiate_control_projections
    that is called in _complete_init_of_partially_initialized_nodes

Note: still need to set  self.needs_update_controller to False
      after instantiating state_input_ports and projections to them

* -

* -

* -

* -

* -

* -

* -

* -

* • Passing all test_control tests except test_mode_based_num_estimates

* • Passing all test_control tests

* -

* • optimizationcontrolmechanism.py
  - _update_state_input_ports_for_controller: handle nested input nodes

* -

* • optimizationcontrolmechanism.py
  _update_state_input_ports_for_controller:  fixed bug with > 1 INPUT node in Composition

* • test_show_graph.py: passes all tests

* -

* • test_report.py:  passing all tests

* • Passes all tests!

* -

* -

* • composition.py: reorganize with #region and #enregions

* • composition.py: reorganize with #region and #enregions

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* -

* -

* -

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* -

* • composition.py:
  __init__: move controller to after add_nodes and add_linear_pathway

* -

* - test_control: only test_hanging_control_spec_outer_controller not passing

* -

* -

* -

* -

* -

* -

* • composition.py:
  _instantiate_control_projections:
     weird requirement for double-call to controller._instantiate_control_signal

* • test_paremtercomposition.py:
  restored parameter spec that causes crash ('threshold',Decision2)

* ª Attempt to fix problem with partially overlapping local and ocm control specs

- composition.py
  - _get_control_signals_for_composition:  (see 11/20/21)
      - added (but commented out change) to "if node.controller" to "if not node.controller"
      - changed append to extend
  - _instantiation_control_projection:
      - got rid of try and except double-call to controller._instantiate_control_signals
      -  outdented call to self.controller._activate_projections_for_composition at end

- controlmechanism.py:
    - _check_for_duplicates:  add warning and return duplicates

- optimizationcontrolmechanism._instantiate_control_signals:
    - add call to self.agent_rep._get_control_signals_for_composition() to get local control specs (on mechs in comp)
    - eliminate duplicates with control_signal specs on OCM
    - instantiate local + ocm control_signals

- parameterestimationcomposition.py
  - added context to various calls

* see later commit

* see later commit

* see later commit

* see later commit

* - This branch passes all tests except:
   - test_parameterestimationcomposition
   - test_composition/test_partially_overlapping_control_specs (ADDED IN THIS COMMINT)

- All relevant changes to this branch are marked as "11/21/21."
  However, most are commented out as they break other things.

- The tests above both involve local control specifications (on mechanism within a nested comp)
  and on the OCM for the outer composition, some of which are for the same nested mechs

- Both tests fail with:
   "AttributeError: 'NoneType' object has no attribute '_get_by_time_scale'" (in component.py LINE 3276)
   This may be due to a problem with context setting, since the error is because the modulation Parameter
   of the ControlProjection is returning "None" rather than "multiplicative_param" (when called with get(context)),
   whereas "multiplicative_param" is returned with a call to get() (i.e., with no context specified)

- Most of test_partially_overlapping_control_specs is passed if
   changes marked "11/21/21 NEW" in optimizationcontrolmechanism.py (LINE 1390) are implemented,
   but it does not properly route ControlProjections through parameter_CIMS (see last assert in test).
   Furthermore, test_parameterestimationcompsition fails with the mod param error, even though the
   model has similar structure (i.e., outer composition -- in this case a ParameterEstimationComposition)
   with an OCM that is given control specs that overlap with ones in a nested composition.

- There are also several other things in composition I found puzzling and tried modifying, but that cuased failures:
  - _get_control_signals_for_composition():
      - seems "if node.controller" should be "if **not** node.controller" (emphasis added just for comment)
      - "append" should be "extend"
  - _instantiate_control_projection():
      -  call to self.controller._activate_projections_for_composition (at end of method) should not be indented

* - small mods; don't impact anything relevant to prior commit message

* - small mods; don't impact anything relevant to prior commit message

* - small mods; don't impact anything relevant to prior commit message

* - finished adding formatting regions to composition.py

* -

* • composition.py:
  - rename _check_projection_initialization_status -> _check_controller_initialization_status
  - add _check_nodes_initialization_status(context=context)
    (and calls it with _check_controller_initialization_status)

* • show_graph.py:  addressed bug associated with ocm.allow_direct_probe

* • show_graph.py:  addressed bug associated with ocm.allow_direct_probe

* -

* Composition: add_controller: set METHOD as context source early

* -

* • composition.py
  retore append of control_signals in _instantiate_control_projections()

* • composition.py
  restore append of control_signals in _instantiate_control_projections()

• test_composition.py:
  add test_partially_overlapping_local_and_control_mech_control_specs_in_unnested_and_nested_comp

* • test_partially_overlapping_local_and_control_mech_control_specs_in_unnested_and_nested_comp():
  - added clear_registry() to allow names to be reused in both runs of test

* • composition.py
  docstring:  added projections entry to list of attributes
  - add_controller:  added call to _add_node_aux_components() for controller

* • composition.py
  _add_node_aux_components(): added deletion of item from aux_components if instantiated

* • composition.py
  - comment out _add_node_aux_components() (causing new failures)
  - move _instantiate_control_projections to be with _instantiate_control_projections,
       after self.add_node(self.controller.objective_mechanism (to be more orderly)

* -

* - confirm that it passes all tests exception test_composition/test_partially_overlapping...
  (with addition of _add_aux_components in add_controller commented out)

* • composition.py:  some more fixed to add_controller that now fail only one test:
    - test_agent_rep_assignement_as_controller_and_replacement

* • Passes *all* current tests

* • composition.py:
  - add_controller:  few more minor mods;
  still passes all tests

* -

* -

* -

* • controlmechanism.py:
  - __init__: resrict specification to only one of control, modulatory_signals, or control_signals (synonyms)

* -

* • composition.py:  in progress fix of bug in instantiating shadow projections for ocm.state_input_ports

* • composition.py:
  - _get_original_senders():  added support for nested composition
    needs to be checked for more than one level
    needs to be refactored to be recursive

* • optimizationcontrolmechanism.py
  - _update_state_input_ports_for_controller:  fix invalid_state_features to allow input_CIM of nested comp in agent_rep

* -

* • composition.py
 - _get_original_senders: made recursive

* • test_show_graph.py: update for fixes

* -

* • tests:  passes all in test_show_graph.py and test_report.py

* Passes all tests

* - comment clean-up

* • composition.py
 - add_controller and _get_nested_node_CIM_port:
   added support for forced assignment of NodeRole.OUTPUT for nodes specified in OCM.monitor_for_control,
   but referenced 'allow_probes' attribute still needs to be implemented

* • composition.py, optimizationcontrolmechanism.py:
  allow_probes fully implemented

* • show_graph.py:  fixed bug causing extra projections to OCM

* • composition.py:
  - _update_shadow_projections(): fix handling of deep nesting

* • optimizationcontrolmechanism.py: add agent_rep_type property

* • optimizationcontrolmechanism.py:
  - state_feature_function -> state_feature_functions

* • optimizationcontrolmechanism.py:
  - _validate_params:  validate state_feature_functions
  - _update_state_input_ports_for_controller: implement assignment of state_feature_functions

* -

* -

* • Passes all tests except test_json with 'model_with_control'

* -

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)
  - remove calls to _update_shadows_dict

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)
  - remove calls to _update_shadows_dict

* -

* • test_two_origins_two_input_ports:  crashes on failure of C->B to update

* -

* • composition.py
  - added property shadowing_dict that has shadowing ports as keys and the ports they shadow as values
  - refactored _update_shadowing_projections to use shadowing_dict

* • optimizationcontrolmechanism.py
  - _update_state_input_ports:  modified validations for nested nodes;  still failing some tests

* • optimizationcontrolmechanism.py
  - _update_state_input_ports:
     more careful and informative validation that state_input_ports are in comp or nested comp and are INPUT nodes thereof;
     passes all tests except test_two_origins_two_input_ports as before

* • composition.py
  _get_invalid_aux_components():  defer all shadow projections until _update_shadow_projections

* • composition.py
  _get_invalid_aux_components():  bug fix in test for shadow projections

* Port: _remove_projection_to_port: don't reduce variable below length 1

even ports with no incoming projections have variable at least length 1

* • composition.py
  add_node(): marked (but haven't removed) code block instantiating shadow_projections
   that seems now to be redundant with _update_shadow_projection

* • show_graph.py
  - _assign_cim_components: supress showing projections not in composition

* • composition.py:
  _analyze_graph():  add extra call to _determine_node_roles after _update_shadow_projections
  _run():  moved block of code at beginning initializing scheduler to after
           _complete_init_of_partially_initialized_nodes and _analyze_graph()

• show_graph.py
  - add test to all loops on projections:  "if proj in composition.projection"

* • show_graph.py
  - add show_projections_not_in_composition option for debugging

* • composition.py
  _update_shadow_projections(): delete unused shadow projections and corresponding ports

* • composition.py
  _update_shadow_projections(): fix bug in deletion of unused shadow projections and ports

• test_show_graph:  tests failing, need mods to accomodate changes

* • composition.py:
  _analyze_graph():  add extra call to _determine_node_roles after _update_shadow_projections
  _run():  moved block of code at beginning initializing scheduler to after
           _complete_init_of_partially_initialized_nodes and _analyze_graph()

• show_graph.py
  - add test to all loops on projections:  "if proj in composition.projection"

* • show_graph.py
  fixes; now passes all show_graph tests

* -

* • composition.py
  _update_shadow_projections:  raise error for attempt to shadow INTERNAL Node of nested comp

* -

* -

* • test_composition.py
  implemented test_shadow_nested_nodes that tests shadowing of nested nodes

* -

* -

* -

* -

* • optimizationcontrolmechanism.py: docstring mods

* -

* -

* • optimizationcontrolmechanism.py
  docstring:  add figure

* • optimizationcontrolmechanism.py
  docstring:  rev figure

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • optimizationcontrolmechanism.py - more docstring edits

* • optimizationcontrolmechanism.py - more docstring edits

* -

* -

* -

* -

* -

* -

* • optimizationcontrolmechanism.py:  add random_params

* -

* -

* -

* -

* • composition.py
  - add random_variable property that returns all Components with seed Parameters
• optimizationcontrolmechanism.py:
  - add random_variable that allows Components to be specified for randomization across num_estimates

* -

Co-authored-by: jdcpni <pniintel55>
Co-authored-by: Jan Vesely <[email protected]>
Co-authored-by: Katherine Mantel <[email protected]>

docs/source/OptimizationControlMechanism.rst

@@ -4,4 +4,4 @@ OptimizationControlMechanism
 .. automodule:: psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism
    :members:
    :private-members:
-   :exclude-members: Linear, random, Parameters
+   :exclude-members: Linear, random, Parameters, OptimizationControlMechanismError

psyneulink/core/components/mechanisms/modulatory/control/controlmechanism.py

@@ -1116,7 +1116,7 @@ class Parameters(ModulatoryMechanism_Base.Parameters):
                     :read only: True
 
                 output_ports
-                    see `output_ports <ControlMechanism.output_ports>`
+                    see `output_ports <Mechanism_Base.output_ports>`
 
                     :default value: None
                     :type: