Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/ocm/state features #2301

Merged
merged 334 commits into from
Jan 31, 2022
Merged

Feat/ocm/state features #2301

merged 334 commits into from
Jan 31, 2022

Conversation

jdcpni
Copy link
Collaborator

@jdcpni jdcpni commented Jan 31, 2022

• optimizationcontrolmechanism.py:

  • _parse_state_feature_specs():
    • refactor to construct state_features dict from _state_feature_specs_parsed and _specified_input_nodes_in_order
    • include all agent_rep INPUT Nodes in state_features, with None assigned as value of unspecified Nodes
  • docstring update for above features, and support of set spec

• test_control.py:

  • test_ocm_state_feature_specs_and_warnings_and_errors():
    • add test for set_spec and set_spec_short
    • add tests for state_feature_values

jdcpni and others added 30 commits November 18, 2021 12:46
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm
  __init__: move controller to after add_nodes and add_linear_pathway
  _instantiate_control_projections:
     weird requirement for double-call to controller._instantiate_control_signal
  restored parameter spec that causes crash ('threshold',Decision2)
…trol specs

- composition.py
  - _get_control_signals_for_composition:  (see 11/20/21)
      - added (but commented out change) to "if node.controller" to "if not node.controller"
      - changed append to extend
  - _instantiation_control_projection:
      - got rid of try and except double-call to controller._instantiate_control_signals
      -  outdented call to self.controller._activate_projections_for_composition at end

- controlmechanism.py:
    - _check_for_duplicates:  add warning and return duplicates

- optimizationcontrolmechanism._instantiate_control_signals:
    - add call to self.agent_rep._get_control_signals_for_composition() to get local control specs (on mechs in comp)
    - eliminate duplicates with control_signal specs on OCM
    - instantiate local + ocm control_signals

- parameterestimationcomposition.py
  - added context to various calls
jdcpni and others added 26 commits January 26, 2022 23:24
  _parse_state_feature_specs() & _validate_state_features(): add mention of missing nodes to warning / error messages
  add _get_nodes_not_in_agent_rep()
  - _parse_state_feature_specs():  enforce list spec for agent_rep_type == COMPOSITION_FUNCTION_APPROXIMATOR
  - add_node():  add call to _analyze_graph if not called from another Composition add method
  - add_node():
    - comment out call to _analyze_graph:  crashes in test_learning_output_shape() with ExecuteMode.LLVM
• optimizationcontrolmechanism.py
  - state_features: clean up
  - _parse_state_feature_sepcs():
    - removed argss:  directly reference self.state_feature_specs and self.state_feature_functions
  - _parse_state_feature_values_from_variable()
    - move to method on Class
    - populate with default variable for INPUT Nodes not specified as state_features
…nUniversity/PsyNeuLink into feat/ocm/state_features

� Conflicts:
�	psyneulink/core/components/mechanisms/modulatory/control/optimizationcontrolmechanism.py
  - remove _parse_state_feature_values_from_variable(): removed
  - add _state_feature_values_getter()
(WITH VALUE OF NONE FOR ONES NOT SPECIFIED IN state_feature_specs)

• optimizationcontrolmechanism.py
  - _state_feature_values_getter(): modify to handle calls in init and validation
(WITH VALUE OF NONE FOR ONES NOT SPECIFIED IN state_feature_specs)
(WITH VALUE OF NONE FOR ONES NOT SPECIFIED IN state_feature_specs)
- PASSING test_ocm_state_feature_specs_and_warnings_and_errors()
- WORKING on test_partial_deferred_init()
• optimizationcontrolmechanism.py:
  - Mark potential mods for making state_feature_specs a Parameter (marked with 1/30/22)
    (with self.state_feature_specs that preserves user specifications)
  - implement self._state_feature_specs_parsed (in place of state_feature_specs Parameter for now)
  - restore self._specified_input_nodes_in_order (in place of state_feature_specs Parameter for now)
  - state.features property constructs dict from above attributes
• optimizationcontrolmechanism.py:
  - Mark potential mods for making state_feature_specs a Parameter (marked with 1/30/22)
    (with self.state_feature_specs that preserves user specifications)
  - implement self._state_feature_specs_parsed (in place of state_feature_specs Parameter for now)
  - restore self._specified_input_nodes_in_order (in place of state_feature_specs Parameter for now)
  - state.features property constructs dict from above attributes
  - test_ocm_state_feature_specs_and_warnings_and_errors(): add tests for state_feature_values
@github-actions
Copy link

This PR causes the following changes to the html docs (ubuntu-latest-3.7-x64):

diff -r docs-base/OptimizationControlMechanism.html docs-head/OptimizationControlMechanism.html
219,220c219,220
< <li><p><a class="reference internal" href="#optimizationcontrolmechanism-model-free"><span class="std std-ref">Model-Free&quot; Optimization</span></a></p></li>
< <li><p><a class="reference internal" href="#optimizationcontrolmechanism-model-based"><span class="std std-ref">Model-Based&quot; Optimization</span></a></p></li>
---
> <li><p><a class="reference internal" href="#optimizationcontrolmechanism-model-free"><span class="std std-ref">&quot;Model-Free&quot; Optimization</span></a></p></li>
> <li><p><a class="reference internal" href="#optimizationcontrolmechanism-model-based"><span class="std std-ref">Model-Based Optimization</span></a></p></li>
232c232,239
< <li><p><a class="reference internal" href="#optimizationcontrolmechanism-state-features-arg"><span class="std std-ref">State Features</span></a></p></li>
---
> <li><dl class="simple">
> <dt><a class="reference internal" href="#optimizationcontrolmechanism-state-features-arg"><span class="std std-ref">State Features</span></a></dt><dd><ul>
> <li><p><a class="reference internal" href="#optimizationcontrolmechanism-agent-rep-composition"><span class="std std-ref">agent_rep Composition</span></a></p></li>
> <li><p><a class="reference internal" href="#optimizationcontrolmechanism-agent-rep-cfa"><span class="std std-ref">agent_rep CompositionFunctionApproximator</span></a></p></li>
> </ul>
> </dd>
> </dl>
> </li>
284a292
> <li><p><a class="reference internal" href="#optimizationcontrolmechanism-execution-timing"><span class="std std-ref">Timing of Execution</span></a></p></li>
344c352
< <figure class="align-default" id="id5">
---
> <figure class="align-default" id="id4">
354c362
< <a class="reference internal" href="CompositionFunctionApproximator.html"><span class="doc">CompositionFunctionApproximator</span></a> as the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a>.</span><a class="headerlink" href="#id5" title="Permalink to this image">¶</a></p>
---
> <a class="reference internal" href="CompositionFunctionApproximator.html"><span class="doc">CompositionFunctionApproximator</span></a> as the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a>.</span><a class="headerlink" href="#id4" title="Permalink to this image">¶</a></p>
407,427c415,449
< <li><p><strong>state_features</strong> – specifies the values provided by the OptimizationControlMechanism as the input to the
< <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a> when used, together with a selected <a class="reference internal" href="ControlMechanism.html#psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation" title="psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">control_allocation</span></code></a>, to estimate or predict the Composition’s <a class="reference internal" href="ControlMechanism.html#psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.net_outcome" title="psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.net_outcome"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">net_outcome</span></code></a>.  These are used to construct the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a> for the OptimizationControlMechanism, the <a class="reference internal" href="InputPort.html#psyneulink.core.components.ports.inputport.InputPort.value" title="psyneulink.core.components.ports.inputport.InputPort.value"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">values</span></code></a>
< of which are assigned to <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span></code></a> and provided to
< the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a> as its input when it is evaluated.  Accordingly, the
< specification requirements for <strong>state_features</strong> depend on whether the
< <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a> is a <a class="reference internal" href="Composition.html"><span class="doc">Composition</span></a> or a <a class="reference internal" href="CompositionFunctionApproximator.html"><span class="doc">CompositionFunctionApproximator</span></a>:</p>
< <ul id="optimizationcontrolmechanism-agent-rep-composition">
< <li><p><em>agent_rep is a Composition</em> – the <strong>state_features</strong> specify the inputs to the Composition when it is executed
< by the OptimizationControlMechanism to <a class="reference internal" href="#optimizationcontrolmechanism-evaluation"><span class="std std-ref">evaluate</span></a> its performance.
< If <strong>state_features</strong> is not specified, this is done automatically by constructing a set of <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a> that <a class="reference internal" href="InputPort.html#inputport-shadow-inputs"><span class="std std-ref">shadow the input</span></a> to every
< <a class="reference internal" href="InputPort.html"><span class="doc">InputPort</span></a> of every <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.NodeRole.INPUT" title="psyneulink.core.compositions.composition.NodeRole.INPUT"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">INPUT</span></code></a> <a class="reference internal" href="Composition.html#composition-nodes"><span class="std std-ref">Node</span></a> of the Composition assigned as
< the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a>.  In this case, if <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.Composition.controller_mode" title="psyneulink.core.compositions.composition.Composition.controller_mode"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">controller_mode</span></code></a> of the Composition for which the OptimizationControlMechanism is the <a class="reference internal" href="Composition.html#composition-controller"><span class="std std-ref">controller</span></a> is set to <em>AFTER</em> (the default), the <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.Composition.input_values" title="psyneulink.core.compositions.composition.Composition.input_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">input</span></code></a> to
< the Composition on the current trial is used as its input to the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a> for the optimization process; if the <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.Composition.controller_mode" title="psyneulink.core.compositions.composition.Composition.controller_mode"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">controller_mode</span></code></a> is <em>BEFORE</em>, then the inputs from the previous trial are used.</p>
< <p id="optimizationcontrolmechanism-state-features-explicit-specification">The <strong>state_features</strong> argument can also be specified explicitly, using the formats described below.  This is
< useful if the values of the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a> are something
< other than the inputs to the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a>, or if different functions need
< to be assigned to different <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a> used to generate
< the corresponding <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span> <span class="pre">state_feature_values</span></code></a>
< (see <a class="reference internal" href="#optimizationcontrolmechanism-state-feature-functions-arg"><span class="std std-ref">below</span></a>). However, doing so overrides the automatic
< assignment of all state_features, and so a complete and appropriate set of specifications must be provided
< (see note below).</p>
---
> <li><p><strong>state_features</strong> – specifies the values provided by the OptimizationControlMechanism as the input to its
> <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a>'s <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.Composition.evaluate" title="psyneulink.core.compositions.composition.Composition.evaluate"><code class="xref any py py-meth docutils literal notranslate"><span class="pre">evaluate</span></code></a> method, together
> with a selected <a class="reference internal" href="ControlMechanism.html#psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation" title="psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">control_allocation</span></code></a>, when that is called to estimate
> or predict the Composition’s <a class="reference internal" href="ControlMechanism.html#psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.net_outcome" title="psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.net_outcome"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">net_outcome</span></code></a>. These are used to construct the
> <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a> for the OptimizationControlMechanism,
> the <a class="reference internal" href="InputPort.html#psyneulink.core.components.ports.inputport.InputPort.value" title="psyneulink.core.components.ports.inputport.InputPort.value"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">values</span></code></a> of which are assigned as the <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span></code></a> and provided to the <strong>predicted_inputs</strong> argument of the
> <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.Composition.evaluate" title="psyneulink.core.compositions.composition.C
...

See CI logs for the full diff.

@lgtm-com
Copy link
Contributor

lgtm-com bot commented Jan 31, 2022

This pull request introduces 2 alerts when merging ab41e77 into 3458e64 - view on LGTM.com

new alerts:

  • 2 for Unused local variable

@jdcpni jdcpni merged commit 3a8bf9d into devel Jan 31, 2022
@jdcpni jdcpni deleted the feat/ocm/state_features branch January 31, 2022 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants