Test/compositon/ validate input dict keys (#2365)

* • optimizationcontrolmechanism.py: docstring mods

* • composition.py:
  - add allow_probes and exclude_probes_from_output

* • composition.py:
  - docstring mods re: allow_probes

• optimizationcontrolmechanism.py:
  - allow_probes:  eliminate DIRECT setting
  - remove _parse_monitor_for_control_input_ports (no longer needed without allow_probes=DIRECT)

* • composition.py:
  - change "exclude_probes_from_output" -> "include_probes_in_output"

* • composition.py:
  - docstring mods re: allow_probes and include_probes_in_output

* • composition.py:
  - docstring mods re: allow_probes and include_probes_in_output

* • controlmechanism.py:
  - add allow_probes handling (moved from OCM)

• optimizationcontrolmechanism.py:
  - move allow_probes to controlmechanism.py

• composition.py:
  - refactor handling of allow_probes to permit for any ControlMechanism

• objectivemechanism.py:
  - add modulatory_mechanism attribute

* • controlmechanism.py:
  - add allow_probes handling (moved from OCM)

• optimizationcontrolmechanism.py:
  - move allow_probes to controlmechanism.py

• composition.py:
  - refactor handling of allow_probes to permit for any ControlMechanism
  - add _handle_allow_probes_for_control() to reconcile setting on Composition and ControlMechanism

• objectivemechanism.py:
  - add modulatory_mechanism attribute

* • composition.py
  add assignment of learning_mechanism to objective_mechanism.modulatory_mechanism for add_learning methods

* • docstring mods

* -

* -

* • optimizationcontrolmechanism.py: docstring revs

* -

* -

* • test_composition.py:
  - add test_unnested_PROBE
  - add test_nested_PROBES
    TBD: test include_probes_in_output

* -

* • composition.py
  - add_node():  support tuple with required_role

* -

* • composition.py:
  - _determine_node_roles:
     fix bug in which nested comp was prevented from being an OUTPUT Node if,
     in addition to Nodes that qualifed as OUTPUT, it also had nodes that projected
     to Nodes in an outer comp (making it look like it was INTERNAL)

* -

* • composition.py:
  - add_node(): enforce include_probes_in_output = True for nested Compositions
  - execute():
    - replace return of output_value with get_output_value()

* -

* • CompositionInterfaceMechanism.rst:
  - correct path ref

• compositioninterfacemechanism.py:
  - docstring fixes

* • optimizationcontrolmechanism.py:
  - docstring edits

* -

* -

* -

* -

* -

* -

* -

* -

* • composition.py, optimizationcontrolmechanism.py:
  _build_predicted_input_dict():  support partial specification of INPUT Nodes for nested Compositions of agent_rep

* • test_control.py:
  - test_nested_composition_as_agent_rep(): PASSES ALL TESTS

* -

* • optimizationcontrolmechanism.py:
  - _parse_state_feature_specs:  support nested composition in dict spec

* • test_control.py
  - test_ocm_state_feature_specs_and_warnings_and_errors():
    - modified to accomodate nested INPUT Node specs
    - PASSES ALL TESTS

* • composition.py:
  - add _nodes_added attribute
  - add_nodes(): set _nodes_added attribute upon completition
  - _analyze_graph(): add call _determine_node_roles() if _nodes_added is True

* • composition.py:
 - _nodes_added -> needs_determine_node_roles

• optimizationcontrolmechanism.py:
  - _get_agent_rep_input_nodes():
    call _determine_node_roles if agent_rep.needs_determine_node_roles is True

* • optimizationcontrolmechanism.py:
  - _parse_state_feature_specs():  consolidate handling of various formats

* • optimizationcontrolmechanism.py: docstring updates

* • optimizationcontrolmechanism.py:
  _get_agent_rep_input_nodes() -> _get_agent_rep_input_receivers()
  _parse_state_feature_specs(): standardize on InputPorts rather than Nodes

* -

* -

* -

* -

* • composition.py:
  - _instantiate_input_dict(): refactor to accept inputs for InputPorts of nested Nodes
  - add helper method _get_external_cim_input_port()
  - TBD:
    - refactor _build_predicted_inputs_dict to support above

* -

* • composition.py:
  - _instantiate_input_dict(): now generates properly formatted values in input_dict for InputPort specs

* • composition.py:
  - _build_predicted_inputs_dict(): refactored to construct inputs_dict with InputPorts as keys

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • optimizationcontrolmechanism.py:
  - refactor state_feature_values as dict that can be used as
    predicted_inputs / feature_values arg of evaluate() (and inputs arg of run)

• composition.py:
  - remove _build_predicted_inputs_dict(),
    since state_feature_values now formatted properly as input

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • composition.py:
  - _instantiate_inputs_dict(): BEFORE REFACTORING FOR MISSING PORTS OF NESTED COMP

* • composition.py:
  - _instantiate_inputs_dict(): BEFORE REFACTORING FOR MISSING PORTS OF NESTED COMP

* • composition.py:
  - _instantiate_inputs_dict(): BEFORE REFACTORING FOR MISSING PORTS OF NESTED COMP

* • composition.py:
  - _instantiate_inputs_dict(): AFTER REFACTORING FOR MISSING PORTS OF NESTED COMP

* -

* -

* -

* -

* • test_composition.py:
  - refactor test_input_type_equivalence()

* - PASSES test_control

* - merged from devel
  missing dependencies

* -

* -

* -

* • update pandas req to 1.4.1

* • composition.py:
  - _instantiate_inputs_dict(): refactored to consolidate treatment of nested Nodes

* -

* -

* • composition.py
  - _instantiate_input_dict():  handle ports with diff numbers of trials specified

* • composition.py
  - _instantiate_input_dict():  handle ports with diff numbers of trials specified

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • composition.py:
  - _parse_names_in_inputs(): support Port.full_name as keys in inputs dict

• mechanism.py:
  - Mechanism_Base: replaced input_labels and output_labels with labeled_input_values and labeled_output_values respectively

• port.py, inputport.py, outputport.py, parameterport.py:
  - Port_Base: impelment labeled_value property (and value_label alias) that returns calls get_label()
  - deleted label attributes of InputPort and OutputPort (replaced by Port_Base.labeled_value)
  - added get_label() stub on ParameterPort with error message

* -

* -

* • conf.py:
  comment out 'sphinx_autodoc_typehints' (crashing for sphinx_autodoc_typehints v.1.17.0)

* • conf.py:
  restore 'sphinx_autodoc_typehints'

• doc_requirements.txt:
  pin sphinx_autodoc_typehints < v.1.16.0;  error on v1.16 and v1.17;  error:
  https://github.com/PrincetonUniversity/PsyNeuLink/runs/5573651890?check_suite_focus=true

* -

* -

* • composition.py:
  - add property: external_input_ports_of_all_input_nodes

• optimizationcontrolmechanism.py:
  - implement state_faature_default
  - add SHADOW_INPUTS as individual spec for state_features

* • composition.py:
  - add property: external_input_ports_of_all_input_nodes

• optimizationcontrolmechanism.py:
  - implement state_faature_default
  - add SHADOW_INPUTS as individual spec for state_features

* -

* -

* -

* -

* -

* -

* -

* -

* • inputport.py:
  - add figure for shadowing

* -

* -

* • processingmechanism.py:
  - __init__(): modify typecheck for input_ports to allow single items

* • optimizationcontrolmechanism.py
  - allow single spec (None, array, tuple, or Components) that is assigned to all INPUT Node InputPorts
  - state_feature_default is assigned to all unspecified INPUT Node InputPorts
    (for a list that is shorter than the number, a dict or a set that has fewer,
     or any that are added to agent_rep after controller is initially constructed and added to Composition)

* -

* -

* • optimizationcontrolmechanism.py: docstring mods

* • optimizationcontrolmechanism.py: docstring mods

* • optimizationcontrolmechanism.py: docstring mods

* -

* -

* -

* -

* -

* • optimizationcontrolmechanism.py:
  - _state_feature_values_getter(): for numeric state_feature, return state_input_port.functio(numeric_value)

* • test_control.py:
  - test_ocm_state_feature_specs_and_warnings_and_errors() - added tests for single state_feature spec

* - composition.py, inputport.py, optimizationcontrolmechanism.py: docstring mods re: "external InputPort"

* -

* -

* -

* • test_control.py:
  - test_state_features_in_nested_composition_as_agent_rep():
    - add tests for single state_feature specs
    - add tests for INPUT Node with more than on InputPort

* -

* -

* -

* • optimizationcontrolmechanism.py:
  - state_features, get_state_feature_source:  simplify

* -

* -

* • optimizationcontrolmechanism.py:
 - state_features, get_state_feature_source: simplify

* • optimizationcontrolmechanism.py: IN PROGRESS
  - _specified_INPUT_Node_InputPorts_in_order:  use name (str) of items not yet in agent_rep rather than 'None'
  - state_features & state_feature_values:
    - use name (str) as key if INPUT Node InputPort is not yet in agent_rep
    - use name (str) if source (spec) is not yet in Composition

* -

* • optimizationcontrolmechanism.py: IMPLEMENTED
  - state_features:
    - convert all keys and values to port.full_name
    - if not yet in comp or agent_rep, add "DEFERRED..."

* • optimizationcontrolmechanism.py:
  PASSES deferred_init and partial_deferred_init tests

* • optimizationcontrolmechanism.py:
  - state_feature_values: strings for missing keys or values

* • test_control.py: PASSING test_deferred and test_partial_deferred

* • test_control.py: PASSING test_deferred and test_partial_deferred

* • test_control.py: PASSING test_deferred and test_partial_deferred

* • test_control.py: PASSING test_ocm_state_feature_specs_and_warnings_and_errors

* • test_control.py: PASSING all state_feature tests

* • composition.py:
  add _is_in_composition method

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • optimzationcontrolmechanism.py:  fix figure

* • optimzationcontrolmechanism.py:  fix figure

* • optimizationcontrolmechanism.py:
  - fix bug in which state_feature_default = None was being ignored
  - fix bug in which deferred nodes were not being assigned state_feature_default

* -

* • optimizationcontrolmechanism.py:
  - fix bug in which state_feature_default = None was being ignored
  - fix bug in which deferred nodes were not being assigned state_feature_default

* • test_control.py:
  - test_deferred_init, test_partial_deferred_init():
    added tests for controller.state_input_ports path_afferents

* -

* -

* • optimizationcontrolmechanism.py:
  - _parse_state_feature_specs:
    - fixed bug in handling of lists of numeric specs
    - renamed state_input_ports for numeric specs

* • registry.py:
  - rename_instance_in_registry(): finish implementing (including **new_name** arg)

• optimizationcontrolmechanism.py:
  - _update_state_features_dict:  rename deferred state_input_ports

* • registry.py:
  - rename_instance_in_registry(): finish implementing (including **new_name** arg)

• optimizationcontrolmechanism.py:
  - _update_state_features_dict:  rename deferred state_input_ports

* -

* • optimizationcontrolmechanism.py, test_control.py
  - standarize prefixes for state_input_ports

* • optimizationcontrolmechanism.py, test_control.py
  - standarize prefixes for state_input_ports

* -

* -

* • test_show_graph remaining

* • mechanism.py:
  - _show_structure(): fix parameterport label bug

* -

* -

* -

* -

* • composition.py:
  _validate_input_keys(): add method to validate keys of inputs to run() method

* • test_composition.py:
  - test_inputs_key_errors()

* • test_composition.py:
  - test_inputs_key_errors():  add test for running nested composition with inputs

* -

* • composition.py:
  - execute(): allow direct call to nested composition with inputs

* -

* -

Co-authored-by: jdcpni <pniintel55>

psyneulink/core/components/mechanisms/mechanism.py

@@ -2406,11 +2406,10 @@ def execute(self,
         """
 
         if self.initialization_status == ContextFlags.INITIALIZED:
-            context.string = "{} EXECUTING {}: {}".format(context.source.name,self.name,
-                                                               ContextFlags._get_context_string(
-                                                                       context.flags, EXECUTION_PHASE))
+            context.string = f"{context.source.name} EXECUTING {self.name}: " \
+                             f"{ContextFlags._get_context_string(context.flags, EXECUTION_PHASE)}."
         else:
-            context.string = "{} INITIALIZING {}".format(context.source.name, self.name)
+            context.string = f"{context.source.name} INITIALIZING {self.name}."
 
         if context.source is ContextFlags.COMMAND_LINE:
             self._initialize_from_context(context, override=False)

psyneulink/core/components/mechanisms/modulatory/control/optimizationcontrolmechanism.py

@@ -1172,11 +1172,6 @@ def _state_feature_values_getter(owning_component=None, context=None):
            len(specified_INPUT_Node_InputPorts) == \
            owning_component.num_state_input_ports
 
-    # If OptimizationControlMechanism is still under construction, use items from input_values as placemarkers
-    # if context.source == ContextFlags.CONSTRUCTOR:
-    #     return {k:v for k,v in zip(specified_INPUT_Node_InputPorts,
-    #                                owning_component.input_values[owning_component.num_outcome_input_ports:])}
-
     # Construct state_feature_values dict
     state_feature_values = {}
     for i in range(owning_component.num_state_input_ports):

psyneulink/core/compositions/composition.py

@@ -4071,7 +4071,8 @@ def add_node(self, node, required_roles=None, context=None):
 
         self._need_check_for_unused_projections = True
 
-        # # MODIFIED 1/27/22 NEW: FIX - BREAKS test_learning_output_shape() in ExecuteMode.LLVM
+        # # MODIFIED 1/27/22 NEW - FIX - BREAKS test_learning_output_shape() in ExecuteMode.LLVM
+        #                                [3/25/22] STILL NEEDED (e.g., FOR test_inputs_key_errors()
         # if context.source != ContextFlags.METHOD:
         #     # Call _analyze_graph with ContextFlags.METHOD to avoid recursion
         #     self._analyze_graph(context=Context(source=ContextFlags.METHOD))
@@ -8454,6 +8455,9 @@ def evaluate(
             buffer_animate_state = self._animate
 
         # Run Composition in "SIMULATION" context
+        # # MODIFIED 3/28/22 NEW:
+        # context.source = ContextFlags.COMPOSITION
+        # MODIFIED 3/28/22 END
         context.add_flag(ContextFlags.SIMULATION_MODE)
         context.remove_flag(ContextFlags.CONTROL)
 
@@ -9269,7 +9273,11 @@ def _validate_execution_inputs(self, inputs):
     #                                           EXECUTION
     # ******************************************************************************************************************
 
+    # MODIFIED 3/28/22 OLD:
     @handle_external_context()
+    # # MODIFIED 3/28/22 NEW:
+    # @handle_external_context(source = ContextFlags.COMMAND_LINE)
+    # MODIFIED 3/28/22 END
     def run(
             self,
             inputs=None,
@@ -9527,7 +9535,9 @@ def run(
             trials.
 
         """
+        # MODIFIED 3/28/22 OLD:
         context.source = ContextFlags.COMPOSITION
+        # MODIFIED 3/28/22 END
         execution_phase = context.execution_phase
         context.execution_phase = ContextFlags.PREPARING
 
@@ -9974,6 +9984,9 @@ def learn(
         runner = CompositionRunner(self)
 
         context.add_flag(ContextFlags.LEARNING_MODE)
+        # # MODIFIED 3/28/22 NEW:
+        # context.source = ContextFlags.COMPOSITION
+        # MODIFIED 3/28/22 END
         # # FIX 5/28/20
         # context.add_flag(ContextFlags.PREPARING)
         # context.execution_phase=ContextFlags.PREPARING
@@ -10220,11 +10233,15 @@ def execute(
                 # These are meant to be assigned in run method;  needed here for direct call to execute method
                 self._animate = False
 
-            # KAM Note 4/29/19
+            # IMPLEMENTATION NOTE:
+            # KAM 4/29/19
             # The nested var is set to True if the Composition is nested in another Composition, otherwise False
             # Later on, this is used to determine:
             #   (1) whether to initialize from context
             #   (2) whether to assign values to CIM from input dict (if not nested) or simply execute CIM (if nested)
+            # JDC 3/28/22:
+            #    This currently prevents a Composition that is nested within another to be tested on its own
+            #    Would be good to figure out a way to accomodate that
             nested = False
             if len(self.input_CIM.path_afferents) > 0:
                 nested = True
@@ -10390,15 +10407,32 @@ def execute(
                 #        but node execution of nested compositions with
                 #        outside control is not supported yet.
                 assert not nested or len(self.parameter_CIM.afferents) == 0
+
             elif nested:
-                # check that inputs are specified - autodiff does not in some cases
-                if ContextFlags.SIMULATION_MODE in context.runmode and inputs is not None:
-                    self.input_CIM.execute(build_CIM_input, context=context)
+
+                # MODIFIED 3/28/22 CURRENT:
+                # IMPLEMENTATION NOTE: context.string set in Mechanism.execute
+                direct_call = (f"{context.source.name} EXECUTING" not in context.string)
+                # MODIFIED 3/28/22 NEW:
+                # direct_call = (context.source == ContextFlags.COMMAND_LINE)
+                # MODIFIED 3/28/22 END
+                simulation = ContextFlags.SIMULATION_MODE in context.runmode
+                if simulation or direct_call:
+                    # For simulations, or direct call to nested Composition (e.g., from COMMAND_LINE to test it)
+                    #  assign inputs if they not provided (e.g., # autodiff)
+                    if inputs is not None:
+                        self.input_CIM.execute(build_CIM_input, context=context)
+                    else:
+                        self.input_CIM.execute(context=context)
                 else:
-                    assert inputs is None, f"Input provided to nested Composition {self.name}; run() method should " \
-                                           f"only be called on outer-most composition within which it is nested."
+                    # regular run (DEFAULT_MODE) of nested Composition called from enclosing Composition,
+                    #    so inputs should be None, and be assigned from nested Composition's input_CIM
+                    assert inputs is None,\
+                        f"Input provided to a nested Composition {self.name} in call from outer composition."
                     self.input_CIM.execute(context=context)
+
                 self.parameter_CIM.execute(context=context)
+
             else:
                 self.input_CIM.execute(build_CIM_input, context=context)
 

tests/composition/test_composition.py

@@ -28,7 +28,7 @@
 from psyneulink.core.components.ports.modulatorysignals.controlsignal import ControlSignal, CostFunctions
 from psyneulink.core.components.projections.modulatory.controlprojection import ControlProjection
 from psyneulink.core.components.projections.pathway.mappingprojection import MappingProjection
-from psyneulink.core.compositions.composition import Composition, CompositionError, NodeRole
+from psyneulink.core.compositions.composition import Composition, NodeRole, CompositionError, RunError
 from psyneulink.core.compositions.pathway import Pathway, PathwayRole
 from psyneulink.core.globals.context import Context
 from psyneulink.core.globals.keywords import \
@@ -2606,6 +2606,82 @@ def test_input_not_provided_to_run(self):
         assert np.allclose(T.parameters.value.get(C), [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
         assert np.allclose(run_result, [[np.array([2.0, 4.0])]])
 
+    input_args = [
+        ('non_input_node',
+         '"The following items specified in the \'inputs\' arg of the run() method for \'Composition-1\' '
+         'are not INPUT Nodes of that Composition (nor InputPorts of them): \'OB\'."'
+         ),
+        ('non_input_port',
+         '"The following items specified in the \'inputs\' arg of the run() method for \'Composition-1\' '
+         'that are not a Mechanism, Composition, or an InputPort of one: \'OA[OutputPort-0]\'."'
+         ),
+        ('nested_non_input_node',
+         '"The following items specified in the \'inputs\' arg of the run() method for \'Composition-1\' '
+         'are not INPUT Nodes of that Composition (nor InputPorts of them): \'IB\'."'
+         ),
+        ('nested_non_input_port',
+        '"The following items specified in the \'inputs\' arg of the run() method for \'Composition-1\' '
+         'that are not a Mechanism, Composition, or an InputPort of one: \'IA[OutputPort-0]\'."'
+         ),
+        ('input_port_and_mech',
+         '"The \'inputs\' arg of the run() method for \'Composition-1\' includes specifications of '
+         'the following InputPorts *and* the Mechanisms to which they belong; '
+         'only one or the other can be specified as inputs to run():  OA[InputPort-0]."'
+         ),
+        ('nested_input_port_and_comp',
+         '"The \'inputs\' arg of the run() method for \'Composition-1\' includes specifications of '
+         'the following InputPorts or Mechanisms *and* the Composition within which they are nested: '
+         '[(\'IA[InputPort-0]\', \'Composition-0\')]."'
+         ),
+        ('nested_mech_and_comp',
+         '"The \'inputs\' arg of the run() method for \'Composition-1\' includes specifications of '
+         'the following InputPorts or Mechanisms *and* the Composition within which they are nested: '
+         '[(\'IA\', \'Composition-0\')]."'
+         ),
+        ('run_nested_with_inputs', None)
+    ]
+    @pytest.mark.parametrize('input_args', input_args, ids=[x[0] for x in input_args])
+    def test_inputs_key_errors(self, input_args):
+
+        condition = input_args[0]
+        expected_error_text = input_args[1]
+
+        ia = ProcessingMechanism(name='IA')
+        ib = ProcessingMechanism(name='IB')
+        oa = ProcessingMechanism(name='OA')
+        ob = ProcessingMechanism(name='OB')
+        icomp = Composition([ia, ib])
+        ocomp = Composition([oa, ob])
+        ocomp.add_node(icomp)
+
+        if condition in {'nested_non_input_node', 'nested_non_input_port'}:
+            X = ia
+            Y = ib
+        else:
+            X = oa
+            Y = ob
+
+        if 'non_input_node' in condition:
+            inputs={X:[1], Y:[1]}
+        elif 'non_input_port' in condition:
+            inputs={X.output_port:[1]}
+        elif condition == 'input_port_and_mech':
+            inputs={X.input_port:[1], X:[1]}
+        elif condition == 'nested_input_port_and_comp':
+            inputs={ia.input_port:[1], icomp:[1]}
+        elif condition == 'nested_mech_and_comp':
+            inputs={ia:[1], icomp:[1]}
+        elif condition == 'run_nested':
+            inputs={ia:[1]}
+
+        if expected_error_text:
+            with pytest.raises(RunError) as error_text:
+                ocomp.run(inputs=inputs)
+            assert expected_error_text in str(error_text.value)
+        else:
+            ocomp._analyze_graph()
+            icomp.run(inputs={ia:[1]})
+
     def test_some_inputs_not_specified(self):
         comp = Composition()