Offshore development updates from cfrontin fork

.github/workflows/python-tests-consolidated.yaml

@@ -203,3 +203,83 @@ jobs:
           cwd=str(path_script.parent),
           env=env,
         )
+
+  find-examples_nb:
+    name: Find all example notebooks
+    needs: [test-unit, test-system]
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Find example python notebooks
+      id: find_notebooks
+      run: |
+        echo "scripts<<EOF"
+        find examples -mindepth 2 -maxdepth 2 -name "*.ipynb" | jq -R -s -c 'split("\n")[:-1]'
+        echo "EOF"
+        echo "scripts<<EOF" >> $GITHUB_OUTPUT
+        find examples -mindepth 2 -maxdepth 2 -name "*.ipynb" | jq -R -s -c 'split("\n")[:-1]' >> $GITHUB_OUTPUT
+        echo "EOF" >> $GITHUB_OUTPUT
+    outputs:
+      scripts: ${{ steps.find_notebooks.outputs.scripts }}
+
+  test-examples_nb:
+    name: Run all example notebooks
+    needs: find-examples_nb
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: [3.12] # ["3.10", "3.11", "3.12", "3.13"]
+        os: [macos-latest, ubuntu-latest, windows-latest]
+        include:
+        - os: ubuntu-latest
+          path: ~/.cache/pip
+        - os: macos-latest
+          path: ~/Library/Caches/pip
+        - os: windows-latest
+          path: ~\AppData\Local\pip\Cache
+        script: ${{fromJson( needs.find-examples_nb.outputs.scripts )}}
+    runs-on: ${{ matrix.os }}
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Get cache dependencies
+      id: cache
+      uses: actions/cache@v4
+      with:
+        path: ${{ matrix.path }}
+        key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
+        restore-keys:
+          ${{ runner.os }}-pip-
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+    - name: Install Ard
+      run: |
+        pip install .[dev]
+    - name: Run examples
+      shell: python
+      run: |
+        import os
+        import pathlib
+        import subprocess
+
+        env = os.environ.copy()
+        env["MPLBACKEND"] = "Agg"  # Non-interactive backend
+
+        path_script = pathlib.Path("${{ matrix.script }}").absolute()
+        print(f"RUNNING {path_script}")
+        subprocess.run(
+          ["jupyter", "nbconvert", "--execute", "--to", "notebook", str(path_script.name)],
+          check=True,
+          cwd=str(path_script.parent),
+          env=env,
+        )

README.md

@@ -1,18 +1,23 @@
-[![CI/CD test suite](https://github.com/WISDEM/Ard/actions/workflows/python-tests-consolidated.yaml/badge.svg?branch=develop)](https://github.com/WISDEM/Ard/actions/workflows/python-tests-consolidated.yaml)
+
 # Ard
 
-**Dig into wind farm design.**
+[![CI/CD test suite](https://github.com/WISDEM/Ard/actions/workflows/python-tests-consolidated.yaml/badge.svg?branch=develop)](https://github.com/WISDEM/Ard/actions/workflows/python-tests-consolidated.yaml/badge.svg?branch=develop)
+
+![Ard logo](assets/logomaker/logo.png)
+
+**Dig in to wind farm design.**
 
 <!-- The (aspirationally) foolproof tool for preparing wind farm layouts. -->
 
 [An ard is a type of simple and lightweight plow](https://en.wikipedia.org/wiki/Ard_\(plough\)), used through the single-digit centuries to prepare a farm for planting.
 The intent of `Ard` is to be a modular, full-stack multi-disciplinary optimization tool for wind farms.
 
-The problem with wind farms is that they are complicated, multi-disciplinary objects.
-They are aerodynamic machines, with complicated control systems, power electronic devices, social and political objects, and the core value (and cost) of complicated financial instruments.
-Moreover, the design of *one* of these aspects affects all the rest!
+Wind farms are complicated, multi-disciplinary systems.
+They are aerodynamic machines (composed of complicated control systems, power electronic devices, etc.), social and political objects, generators of electrical power and consumers of electrical demand, and the core value generator (and cost) of complicated financial instruments.
+Moreover, the design of any *one* of these aspects affects all the rest!
 
-`Ard` seeks to make plant-level design choices that can incorporate these different aspects _and their interactions_ to make wind energy projects more successful.
+`Ard` is a platform for wind farm layout optimization that seeks to enable plant-level design choices that can incorporate these different aspects _and their interactions_ to make wind energy projects more successful.
+In brief, we are designing `Ard` to be: principled, modular, extensible, and effective, to allow resource-specific wind farm layout optimization with realistic, well-posed constraints, holistic and complex objectives, and natural incorporation of multiple fidelities and disciplines.
 
 ## Documentation
 Ard documentation is available at [https://wisdem.github.io/Ard/]()
@@ -45,7 +50,6 @@ For a basic and static installation, type:
 ```shell
 pip install .
 ```
-
 For development (and really for everyone during pre-release), we recommend a full development installation:
 ```shell
 pip install -e .[dev,docs]
@@ -60,59 +64,69 @@ mamba install wisdem -y
 pip install -e .[dev,docs]
 ```
 
-To test the installation, from the `Ard` folder run unit and regression tests:
+## Testing instructions
+
+The installation can be tested comprehensively using `pytest` from the top-level directory.
+The developers also provide some convenience scripts for testing new installations; from the `Ard` folder run unit and regression tests:
 ```shell
 source test/run_local_test_unit.sh
 source test/run_local_test_system.sh
 ```
+These enable the generation of HTML-based coverage reports by default and can be used to track "coverage", or the percentage of software lines of code that are run by the testing systems.
+`Ard`'s git repository includes requirements for both the `main` and `develop` branches to have 80% coverage on unit testing and 50% testing in system testing, which are, respectively, tests of individual parts of `Ard` and "systems" composed of multiple parts.
+Failures are not tolerated in code that is merged onto these branches and code found therein *should* never cause a testing failure if it has been found there.
+If the process of installation and testing fails, please open a new issue [here](https://github.com/WISDEM/Ard/issues).
 
-For user information, in pre-release, we are using some co-developed changes to the `FLORIS` library.
-
-If the installation fails, please open a new issue [here](https://github.com/WISDEM/Ard/issues).
+## Design philosophy
 
-## OptiWindNet
+The design of `Ard` was inspired by two use cases in particular:
+1) systems energy researchers who are focusing on one specific subdiscipline (e.g. layout strategies, social impacts, or aerodynamic modeling) but want to be able to easily keep track of how a change in one discipline impacts the entire value chain down to production, cost, value, and/or societal outcomes of energy or even optimize with respect to these, and
+2) private industry researchers who run business cases and may want to drop in proprietary analysis modules for specific disciplines while preserving some of the open-source modules of `Ard`.
 
-We currently have experimental support for [Mauricio Souza de Alencar's OptiWindNet package for collection system cable path-planning optimization](https://gitlab.windenergy.dtu.dk/TOPFARM/OptiWindNet).
+`Ard` is being developed as a modular tool to enable these types of research queries.
+The goals during the development of `Ard` are to be:
+1) principled:
+   - robustly documented
+   - adhering to [best-practices for code development](https://doi.org/10.2172/2479115)
+2) modular and extensible:
+   - choose the analysis components you want
+   - skip the ones you don't
+   - build yourself the ones we don't have
+3) effective
+    - robustly tested and testable at both unit and system levels
+These principles guide us to implement, using [`OpenMDAO`](https://openmdao.org) as a backbone, a multi-disciplinary design, analysis, and optimization (MDAO) model of the wind farm layout problem, a toolset to accomplish the capability goals of `Ard`, to:
+1) allow optimization of wind farm layouts for specific wind resource profiles
+2) enable the incorporation of realistic but well-posed constraints
+3) target holistic and complex system-level optimization objectives like LCOE and beyond-LCOE metrics
+4) naturally incorporate analyses across fidelities to efficiently integrate advanced simulation
 
 ## Current capabilities
 
-For the alpha pre-release of `Ard`, we have concentrated on optimization of wind plants, starting from a structured layout and optimizing it to minimize the levelized cost of energy, or LCOE.
-This capability is demonstrated for a land-based (LB) wind farm in `examples/LCOE_LB_stack` and tested in an abridged form in `test/system/LCOE_stack/test_LCOE_LB_stack.py`.
-In this example, the wind farm layout is parametrized with two angles, named orientation and skewed, and turbine distancing for rows and columns.
-In the alpha pre-release stage, the constituent subcomponents of these problems are known to work and fully tested; any capabilities not touched in the layout-to-LCOE stack should be treated as experimental.
+For the beta pre-release of `Ard`, we concentrate on optimization problems for wind plants, starting from structured layouts to minimize LCOE.
+This capability is demonstrated for a land-based (LB) wind farm in `examples/01_onshore` and tested in an abridged form in `test/system/ard/api/test_LCOE_LB_stack.py`.
+In this example, the wind farm layout is parametrized with two angles, named orientation and skew, and turbine distancing for rows and columns.
+Additionally, we have offshore examples adjacent to the onshore example in the `examples` subdirectory.
+In the beta pre-release stage, the constituent subcomponents of these problems are known to work and have full testing coverage.
 
-These cases start from a four parameter farm layout, compute landuse area, make FLORIS AEP estimates, compute turbine capital costs, balance-of-station (BOS), and operational costs using WISDEM components, and finally give summary estimates of plant finance figures.
+These cases start from a four parameter farm layout, compute land use area, make FLORIS estimates of annual energy production (AEP), compute turbine capital costs, balance-of-station (BOS), and operational costs elements of NREL's turbine systems engineering tool [WISDEM](https://github.com/wisdem/wisdem), and finally give summary estimates of plant finance figures.
 The components that achieve this can be assembled to either run a single top-down analysis run, or run an optimization.
 
-A second example is in progress to reoptimize the layout of two offshore wind farms, one fixed bottom (OFB) and one floating (OFL).
-Both wind farms are made of the [22 MW reference wind turbine](https://github.com/IEAWindSystems/IEA-22-280-RWT).
-In this example, BOS costs are estimated using the tool [ORBIT](https://github.com/WISDEM/ORBIT).
+# Contributing to `Ard`
 
-## Roadmap to future capabilities
-
-The future development of `Ard` is centered around two user cases:
-1) systems energy researchers who are focusing on one specific subdiscipline (e.g. layout strategies, social impacts, or aerodynamic modeling) but want to be able to easily keep track of how it impacts the entire value chain down to production, cost, and/or value of energy or even optimize with respect to it, and
-2) private industry researchers who are interested in how public-sector research results change when proprietary analysis tools are dropped in and coupled the other tools in a systems-level simulation.
-
-`Ard` is being developed as a modular tool to enable these types of research queries.
-This starts from our research goals, which are that `Ard` should be:
-1) principled: fully documented, and adhering to best-practices for code development
-2) modular and extensible: choose the parts you want, skip the ones you don't, build yourself the ones we don't have
-3) effective: fully tested and testable at the unit and system level, and built with a derivative-forward approach
-
-This, then, allows us to attempt to accomplish the technical goals of `Ard`, to:
-1) allow optimization of wind farm layouts for specific wind resource profiles
-2) target wholistic and complex system-level optimization objectives like LCOE and beyond-LCOE metrics
-3) naturally incorporate multi-fidelity analyses to efficiently integrate physics-resolving simulation
+We have striven towards best-practices documentation and testing for `Ard`.
+Contribution is welcome, and we are happy [to field pull requests from github](https://github.com/WISDEM/Ard/pulls).
+For acceptance, PRs must:
+- be formatted using [`black`](https://github.com/psf/black)
+- not fail any unit tests or system tests
+- achieve coverage criteria for unit & system testing
+- be documented enough for continued maintenance by core `Ard` developers
 
 ## Building Documentation
 
-To build the documentation locally run the following from the top `Ard/` directory.
-
+To build the documentation locally, run the following from the top-level `Ard/` directory:
 ```shell
 jupyter-book build docs/
 ```
-
 You can then open `Ard/docs/_build/html/index.html` to view the docs.
 
 ---

ard/api/interface.py

@@ -107,6 +107,8 @@ def set_up_ard_model(input_dict: Union[str, dict], root_data_path: str = None):
         analysis_options=input_dict["analysis_options"],
     )
 
+    prob.setup()
+
     return prob
 
 
@@ -203,6 +205,7 @@ def set_up_system_recursive(
             if "driver" in analysis_options:
                 Driver = getattr(om, analysis_options["driver"]["name"])
 
+                # handle DOE drivers with special treatment
                 if Driver == om.DOEDriver:
                     generator = None
                     if "generator" in analysis_options["driver"]:
@@ -278,6 +281,4 @@ def set_up_system_recursive(
             units="m",
         )
 
-        prob.setup()
-
     return prob

ard/collection/optiwindnet_wrap.py

@@ -4,7 +4,7 @@
 from optiwindnet.mesh import make_planar_embedding
 from optiwindnet.interarraylib import L_from_site
 from optiwindnet.heuristics import EW_presolver
-from optiwindnet.MILP import solver_factory, ModelOptions
+from optiwindnet.MILP import OWNWarmupFailed, solver_factory, ModelOptions
 
 from . import templates
 
@@ -69,6 +69,9 @@ class OptiwindnetCollection(templates.CollectionTemplate):
     -------
     total_length_cables : float
         the total length of cables used in the collection system network
+    terse_links : np.ndarray
+        a 1D numpy int array encoding the electrical connections of the collection
+        system (tree topology), with length `N_turbines`
 
     Discrete Outputs
     -------
@@ -81,14 +84,12 @@ class OptiwindnetCollection(templates.CollectionTemplate):
         length `N_turbines`
     max_load_cables : int
         the maximum cable capacity required by the collection system
-    terse_links : np.ndarray
-        a 1D numpy int array encoding the electrical connections of the collection
-        system (tree topology), with length `N_turbines`
     """
 
     def initialize(self):
         """Initialization of OM component."""
         super().initialize()
+        self.S_previous: nx.Graph | None = None
 
     def setup(self):
         """Setup of OM component."""
@@ -102,6 +103,13 @@ def setup_partials(self):
             ["x_turbines", "y_turbines", "x_substations", "y_substations"],
             method="exact",
         )
+        self.declare_partials(
+            ["terse_links"],
+            ["x_turbines", "y_turbines", "x_substations", "y_substations"],
+            method="exact",
+            val=0.0,
+            dependent=False,
+        )
 
     def compute(
         self,
@@ -126,20 +134,42 @@ def compute(
         # create planar embedding and set of available links
         P, A = make_planar_embedding(L)
 
-        # presolve
-        S_warm = EW_presolver(A, capacity=max_turbines_per_string)
+        solver = solver_factory(solver_name)
+
+        model_options = self.modeling_options["collection"]["model_options"]
+        # start from previous solution if available, else from heuristic if it fits
+        if self.S_previous is not None:
+            S_warm = self.S_previous
+        elif (
+            model_options.get("topology") == "branched"
+            and model_options.get("feeder_limit") == "unlimited"
+            and model_options.get("feeder_route") == "segmented"
+        ):
+            S_warm = EW_presolver(A, capacity=max_turbines_per_string)
+        else:
+            S_warm = None
+
+        try:
+            solver.set_problem(
+                P,
+                A,
+                max_turbines_per_string,
+                ModelOptions(**model_options),
+                warmstart=S_warm,
+            )
+        except OWNWarmupFailed:
+            # the previous solution is no longer feasible
+            solver.set_problem(
+                P,
+                A,
+                max_turbines_per_string,
+                ModelOptions(**model_options),
+            )
 
         # do the branch-and-bound MILP search
-        solver = solver_factory(solver_name)
-        solver.set_problem(
-            P,
-            A,
-            max_turbines_per_string,
-            ModelOptions(**self.modeling_options["collection"]["model_options"]),
-            warmstart=S_warm,
-        )
         info = solver.solve(**self.modeling_options["collection"]["solver_options"])
         S, G = solver.get_solution()
+        self.S_previous = S
 
         # extract the outputs
         terse_links = np.zeros((T,), dtype=np.int_)
@@ -178,14 +208,14 @@ def compute(
         # pack and ship
         self.graph = G
         discrete_outputs["graph"] = G  # TODO: remove for terse links, below!
-        discrete_outputs["terse_links"] = terse_links
         discrete_outputs["length_cables"] = length_cables
         discrete_outputs["load_cables"] = load_cables
         discrete_outputs["max_load_cables"] = S.graph["max_load"]
         # TODO: remove this assert after enough testing
         assert (
             abs(length_cables.sum() - G.size(weight="length")) < 1e-7
         ), f"difference: {length_cables.sum() - G.size(weight='length')}"
+        outputs["terse_links"] = terse_links
         outputs["total_length_cables"] = length_cables.sum()
 
     def compute_partials(self, inputs, J, discrete_inputs=None):

ard/collection/templates.py

@@ -108,8 +108,8 @@ def setup(self):
 
         # set up outputs for the collection system
         self.add_output("total_length_cables", 0.0, units="m")
+        self.add_output("terse_links", np.full((self.N_turbines,), -1))
         self.add_discrete_output("length_cables", np.zeros((self.N_turbines,)))
-        self.add_discrete_output("terse_links", np.full((self.N_turbines,), -1))
         self.add_discrete_output("load_cables", np.zeros((self.N_turbines,)))
         self.add_discrete_output("max_load_cables", 0.0)
         self.add_discrete_output("graph", None)