Refactor/controlmech and ocm (#2186)

* • component.py
  docstring mod to **size**

* • controlmechanism.py: add support for monitor_for_control in _instantiate_input_ports

* -

* • controlmechanism.py: refactoring to use outcome_input_ports_option

* • controlmechanism.py: refactored to use outcome_input_ports
  following tests not passing:
    - control/test_param_estimation_test_moving_average
        (elfi)
    - test_json:
        test_json_results_equivalence
           (model_with_control_py-comp-{Input: [0.5, 0.123], reward: [20, 20]})
        test_write_json_file
           (model_with_control_py-comp-{Input: [0.5, 0.123], reward: [20, 20]})

* -

* -

* -

* -

* -

* Tests passing

* • controlMechanism.py:
  - added outcome_input_ports_option: SEPARATE, COMBINE, CONCATENATE
  - refactored _outcome_getter to return value of all InputPorts in outcome_input_ports

* -

* • port.py:
  _instantiate_projection_from_port raise PortError if type_match fails
  for projection and parameter value during instantiation

* • port.py:
  _instantiate_projection_from_port raise PortError if type_match fails
  for projection and parameter value during instantiation

* • controlmechanism.py: fixed bugs in use of COMBINE and CONCATENATE

* • controlmechanism.py: fixed more bugs in use of COMBINE and CONCATENATE

* • optimizationcontrolmechanism.py: renamed evaluation_function -> evaluate_agent_rep

* • optimizationfunctions.py:
     add aggregation_function as Parameter
     add num_estimates as argument for _function
     loop over num_estimates in call to _objective_function
         and use aggregation_function to aggregate results

* • optimizationfunctions.py:
     add aggregation_function as Parameter
     add num_estimates as argument for _function
     loop over num_estimates in call to _objective_function
         and use aggregation_function to aggregate results

* -

* • optimizationfunctions.py:
  - implemented randomization_seed_dimension argument
  - implemented num_estimates property (uses randomization_seed_dimension)
• optimizationcontrolmechanism.py:
  - validates that num_estimates (if specified) is consistent with
     number generated by allocation_samples for seed ControlSignal
  - passes randomization_seed_dimension to OptimizationFunction constructor

* -

* -

* -

* • tests/composition: add test_parameterestimationcomposition.py

* • tests/composition: add test_parameterestimationcomposition.py (basic form)

* -

* -

* -

* -

* • optimizationcontrolmechanism.py:
   moved construction of randomization ControlSignal here from PEC
Passes all tests

* -

* -

* • optimizationcontrolmechanism.py:
    renamed same_seed_for_all_parameter_combinations -> same_seed_for_all_allocations
    (left name the same on PEC for UI reasons)
    note: use of parameter by ObjectiveFunction remains to be implemented

* • optimizationcontrolmechanism.py: docstring updates to reflect preceding changes

* • optimizationcontrolmechanism.py: more docstring edits

* -

* -

* -

* -

Co-authored-by: jdcpni <pniintel55>

Scripts/Models (Under Development)/Bustamante_Stroop_XOR_LVOC_Model.py

@@ -181,7 +181,7 @@ def print_weights():
     print('ControlSignal variables: ', [sig.parameters.variable.get(i) for sig in lvoc.control_signals])
     print('ControlSignal values: ', [sig.parameters.value.get(i) for sig in lvoc.control_signals])
     # print('state_features: ', lvoc.get_feature_values(context=c))
-    print('lvoc: ', lvoc.evaluation_function([sig.parameters.variable.get(i) for sig in lvoc.control_signals], context=i))
+    print('lvoc: ', lvoc.evaluate_agent_rep([sig.parameters.variable.get(i) for sig in lvoc.control_signals], context=i))
     # print('time: ', duration)
     print('--------------------')
 
@@ -222,6 +222,6 @@ def print_weights():
 # print('ControlSignal variables: ', [sig.parameters.variable.get(c) for sig in lvoc.control_signals])
 # print('ControlSignal values: ', [sig.parameters.value.get(c) for sig in lvoc.control_signals])
 # # print('state_features: ', lvoc.get_feature_values(context=c))
-# print('lvoc: ', lvoc.evaluation_function([sig.parameters.variable.get(c) for sig in lvoc.control_signals], context=c))
+# print('lvoc: ', lvoc.evaluate_agent_rep([sig.parameters.variable.get(c) for sig in lvoc.control_signals], context=c))
 # print('time: ', duration)
 # print('--------------------')

Scripts/Models (Under Development)/Bustamante_Stroop_XOR_LVOC_Model_VZ.py

@@ -232,7 +232,7 @@ def print_weights():
     print('ControlSignal variables: ', [sig.parameters.variable.get(i) for sig in lvoc.control_signals])
     print('ControlSignal values: ', [sig.parameters.value.get(i) for sig in lvoc.control_signals])
     # print('state_features: ', lvoc.state_feature_values)
-    # print('lvoc: ', lvoc.evaluation_function([sig.parameters.variable.get(i) for sig in lvoc.control_signals], context=i))
+    # print('lvoc: ', lvoc.evaluate_agent_rep([sig.parameters.variable.get(i) for sig in lvoc.control_signals], context=i))
     # print('time: ', duration)
     print('--------------------')
 

psyneulink/core/components/component.py

@@ -1697,16 +1697,8 @@ def _deferred_init(self, **kwargs):
             self._init_args.update(kwargs)
 
             # Complete initialization
-            # MODIFIED 10/27/18 OLD:
             super(self.__class__,self).__init__(**self._init_args)
 
-            # MODIFIED 10/27/18 NEW:  FOLLOWING IS NEEDED TO HANDLE FUNCTION DEFERRED INIT (JDC)
-            # try:
-            #     super(self.__class__,self).__init__(**self._init_args)
-            # except:
-            #     self.__init__(**self._init_args)
-            # MODIFIED 10/27/18 END
-
             # If name was assigned, "[DEFERRED INITIALIZATION]" was appended to it, so remove it
             if DEFERRED_INITIALIZATION in self.name:
                 self.name = self.name.replace("[" + DEFERRED_INITIALIZATION + "]", "")

psyneulink/core/components/mechanisms/mechanism.py

@@ -1380,20 +1380,20 @@ class Mechanism_Base(Mechanism):
 
     projections : ContentAddressableList
         a list of all of the Mechanism's `Projections <Projection>`, composed from the
-        `path_afferents <InputPorts.path_afferents>` of all of its `input_ports <Mechanism_Base.input_ports>`,
+        `path_afferents <Port.path_afferents>` of all of its `input_ports <Mechanism_Base.input_ports>`,
         the `mod_afferents` of all of its `input_ports <Mechanism_Base.input_ports>`,
         `parameter_ports <Mechanism)Base.parameter_ports>`, and `output_ports <Mechanism_Base.output_ports>`,
         and the `efferents <Port.efferents>` of all of its `output_ports <Mechanism_Base.output_ports>`.
 
     afferents : ContentAddressableList
         a list of all of the Mechanism's afferent `Projections <Projection>`, composed from the
-        `path_afferents <InputPorts.path_afferents>` of all of its `input_ports <Mechanism_Base.input_ports>`,
+        `path_afferents <Port.path_afferents>` of all of its `input_ports <Mechanism_Base.input_ports>`,
         and the `mod_afferents` of all of its `input_ports <Mechanism_Base.input_ports>`,
         `parameter_ports <Mechanism)Base.parameter_ports>`, and `output_ports <Mechanism_Base.output_ports>`.,
 
     path_afferents : ContentAddressableList
         a list of all of the Mechanism's afferent `PathwayProjections <PathwayProjection>`, composed from the
-        `path_afferents <InputPorts.path_afferents>` attributes of all of its `input_ports
+        `path_afferents <Port.path_afferents>` attributes of all of its `input_ports
         <Mechanism_Base.input_ports>`.
 
     mod_afferents : ContentAddressableList

psyneulink/core/components/mechanisms/processing/objectivemechanism.py

@@ -833,6 +833,8 @@ def monitored_output_ports_weights_and_exponents(self):
     def monitored_output_ports_weights_and_exponents(self, weights_and_exponents_tuples):
         self.monitor_weights_and_exponents = weights_and_exponents_tuples
 
+# FIX: 11/3/21 -- MOVE THIS TO ControlMechanism, AND INTEGRATE WITH ControlMechanism.validate_monitored_port_spec()
+#                 OR MOVE THAT METHOD TO HERE??
 def _parse_monitor_specs(monitor_specs):
     spec_tuple = namedtuple('SpecTuple', 'index spec')
     parsed_specs = []

psyneulink/core/components/ports/port.py

@@ -1766,7 +1766,12 @@ def _get_receiver_port(spec):
                                                                                               context=context)
                         # Match the projection's value with the value of the function parameter
                         # should be defaults.value?
-                        mod_proj_spec_value = type_match(projection.value, type(mod_param_value))
+                        try:
+                            mod_proj_spec_value = type_match(projection.value, type(mod_param_value))
+                        except TypeError as error:
+                            raise PortError(f"The value for {self.name} of {self.owner.name} ({projection.value}) does "
+                                            f"not match the format ({mod_param_value}) of the Parameter it modulates "
+                                            f"({receiver.owner.name}[{mod_param_name}]).")
                         if (mod_param_value is not None
                             and not iscompatible(mod_param_value, mod_proj_spec_value)):
                             raise PortError(f"Output of {projection.name} ({mod_proj_spec_value}) is not compatible "
@@ -3260,8 +3265,10 @@ def _parse_port_spec(port_type=None,
     #                                 port_dict[OWNER].name, spec_function_value, spec_function))
 
     if port_dict[REFERENCE_VALUE] is not None and not iscompatible(port_dict[VALUE], port_dict[REFERENCE_VALUE]):
-        raise PortError("Port value ({}) does not match reference_value ({}) for {} of {})".
-                         format(port_dict[VALUE], port_dict[REFERENCE_VALUE], port_type.__name__, owner.name))
+        port_name = f"the {port_dict[NAME]}" if (NAME in port_dict and port_dict[NAME]) else f"an"
+        raise PortError(f"The value ({port_dict[VALUE]}) for {port_name} {port_type.__name__} of "
+                        f"{owner.name} does not match the reference_value ({port_dict[REFERENCE_VALUE]}) "
+                        f"used for it at construction.")
 
     return port_dict
 

psyneulink/core/compositions/composition.py

@@ -402,10 +402,10 @@
       - `Composition_Controller_Execution`
 
 
-A Composition can be assigned a `controller <Composition.controller>`.  This is a `ControlMechanism`, or a subclass of
-one, that modulates the parameters of Components within the Composition (including Components of nested Compositions).
-It typically does this based on the output of an `ObjectiveMechanism` that evaluates the value of other Mechanisms in
-the Composition, and provides the result to the `controller <Composition.controller>`.
+A Composition can be assigned a `controller <Composition.controller>`.  This must be an `OptimizationControlMechanism`,
+or a subclass of one, that modulates the parameters of Components within the Composition (including Components of
+nested Compositions). It typically does this based on the output of an `ObjectiveMechanism` that evaluates the value
+of other Mechanisms in the Composition, and provides the result to the `controller <Composition.controller>`.
 
 .. _Composition_Controller_Assignment:
 
@@ -2370,8 +2370,9 @@ def input_function(env, result):
 
 from psyneulink.core import llvm as pnlvm
 from psyneulink.core.components.component import Component, ComponentsMeta
-from psyneulink.core.components.functions.nonstateful.combinationfunctions import LinearCombination, PredictionErrorDeltaFunction
 from psyneulink.core.components.functions.function import is_function_type
+from psyneulink.core.components.functions.nonstateful.combinationfunctions import LinearCombination, \
+    PredictionErrorDeltaFunction
 from psyneulink.core.components.functions.nonstateful.learningfunctions import \
     LearningFunction, Reinforcement, BackPropagation, TDLearning
 from psyneulink.core.components.functions.nonstateful.transferfunctions import Identity
@@ -7140,6 +7141,7 @@ def add_controller(self, controller:ControlMechanism, context=None):
         if invalid_aux_components:
             self._controller_initialization_status = ContextFlags.DEFERRED_INIT
 
+        # FIX: 11/3/21: ISN'T THIS HANDLED IN HANDLING OF aux_components?
         if self.controller.objective_mechanism and self.controller.objective_mechanism not in invalid_aux_components:
             self.add_node(self.controller.objective_mechanism, required_roles=NodeRole.CONTROLLER_OBJECTIVE)
 
@@ -7159,6 +7161,8 @@ def add_controller(self, controller:ControlMechanism, context=None):
         input_cims= [self.input_CIM] + nested_cims
         # For the rest of the controller's input_ports if they are marked as receiving SHADOW_INPUTS,
         #    instantiate the shadowing Projection to them from the sender to the shadowed InputPort
+        # FIX: 11/3/21: BELOW NEEDS TO BE CORRECTED IF OUTCOME InputPort GETS MOVED
+        #               ALSO, IF Non-OCM IS USED AS CONTROLLER, MAY HAVE MORE THAN ONE Inport FOR MONITORING
         for input_port in controller.input_ports[1:]:
             if hasattr(input_port, SHADOW_INPUTS) and input_port.shadow_inputs is not None:
                 for proj in input_port.shadow_inputs.path_afferents:
@@ -7189,7 +7193,7 @@ def add_controller(self, controller:ControlMechanism, context=None):
                 if proj.sender.owner not in nested_cims:
                     proj._activate_for_compositions(self)
 
-        # Check whether controller has input, and if not then disable
+        # Confirm that controller has input, and if not then disable it
         if not (isinstance(self.controller.input_ports, ContentAddressableList)
                 and self.controller.input_ports):
             # If controller was enabled, warn that it has been disabled
@@ -7260,14 +7264,17 @@ def _build_predicted_inputs_dict(self, predicted_input):
         # If this is not a good assumption, we need another way to look up the feature InputPorts
         # of the OCM and know which InputPort maps to which predicted_input value
 
-        if predicted_input is None:
+        no_predicted_input = (predicted_input is None or not len(predicted_input))
+        if no_predicted_input:
             warnings.warn(f"{self.name}.evaluate() called without any inputs specified; default values will be used")
 
+
         nested_nodes = dict(self._get_nested_nodes())
+        # FIX: 11/3/21 NEED TO MODIFY WHEN OUTCOME InputPort IS MOVED
         shadow_inputs_start_index = self.controller.num_outcome_input_ports
         for j in range(len(self.controller.input_ports) - shadow_inputs_start_index):
             input_port = self.controller.input_ports[j + shadow_inputs_start_index]
-            if predicted_input is None:
+            if no_predicted_input:
                 shadowed_input = input_port.defaults.value
             else:
                 shadowed_input = predicted_input[j]