Text-based (YAML) math definition interface

[brynpickering]

It would be useful to be able to define all the mathematics of a linopy model in human-readable text format. This would help with transparency and shareability (if that's a word) of math, as well as making it easier to define new math without needing to be so familiar with Python.

I have developed something that I think would be suitable in the Calliope project. It is a vectorised syntax that is documented. The main ideas are:

1. it works with xarray dataarrays, using a foreach directive to define the array dimensions and a where directive to mask the array so that variables/exprs/constraints can be defined only for a subset.
2. Math expressions do not reference dimensions unless slicing specific elements in the math. That is, you could define p <= p_nom and it would be applied automatically to all entries in the dimensions defined in foreach, or you could define p[snapshot=2012-01-01] <= p_nom * 0.8 and it would slice p to a given snapshot.
3. helper functions exist to deal with more complex cases. An obvious use-case is to roll the timeseries dimension forward/backward on one element so that you can get the equivalent of state_of_charge[t] = state_of_charge[t-1] + p_store[t] - p_discharge[t] by calling state_of_charge = roll(state_of_charge, snapshot=1) + p_store - p_discharge. These are designed as plugins so that power users can add and use new ones if the need arises.
4. It can automatically be rendered as LaTeX math for use in documentation. This means you only document your math once, when defining it in YAML, and it can then be kept up-to-date in the docs automatically.

It may not be fit for all linopy purposes but the fact that it also relies on multi-dim xarrays I think is already a good starting point. I have been exploring how it could also be used for tidy dataframes (if e.g. updating to using pyoframe) and it seems relatively straightforward; I just need to better understand how to apply the where string masking on polars dfs.

To mitigate bad actors injecting code into the expressions, all strings are parsed with pyparsing (ref. where and expression strings).

Examples of use for user-defined math can be seen here.

[coroa]

That would be a wonderful addition especially for making the custom constraints interface of PyPSA declarative (and thus persistable within a saved network) rather than providing only a callback function.

Do you have any thoughts on how to bring in external data (f.ex. an external csv file containing some co2 limits)?