Enable bandwidths for scalars and timeseries

[henhuy]

A new OEDatamodel version shall be developed, which allows bandwidths for scalars and timeseries per scenario.
IMO this needs three implementation changes:

1. value in scalars must be changed to hold bandwidths (allowing fixed, discrete and continuous values)
2. multiple timeseries regarding same region/tech/etc. are allowed, but must be distinguishable by user
3. a concrete instance of the scenario using fixed values and fixed timeseries from scenario with bandwiths must be build

Possible implementations can be discussed here.

[henhuy]

My suggestion for task one (bandwidths in scalars), is to change type of column value (now float) into list-of-floats and to add column bandwith_type (or different name) of type string containing one of fixed/discrete/continuous.

[henhuy]

Regarding task two, we could use tag column for explanations about the timeseries

[henhuy]

Task three could be implemented, by adding three new tables scenario_instance, scalar_instances, timeseries_instances (search for better names!) with following functions:

• scenario_instance simply contains PK and FK to instantiated scenario
• scalar_instances contains columns PK, FK to scenario_instance, FK to scalar (with bandwiths) and value (float) holding a valid value within bandwidth of related scalar entry
• timeseries_instances contains PK, FK to scenario_instance and FK to timeseries

From those tables all needed information for one reference scenario can be gathered...

[jnnr]

Great initiative!
Regarding 1., I suggest to allow for fixed scalars, lists of scalars and distributions. Distributions can be specified by a finite number of parameters, i.e. a uniform distribution between an upper and lower bound, a Gaussian characterized by mean and variance etc. Together with a sampling method, this is enough information to produce a concrete scenario instance.

Similarly for 2., but for timeseries in most of the cases you would have a fixed timeseries or a list of timeseries that can be used to sample from.

[chrwm]

Hi @jnnr
Regarding 1., @srhbrnds, @henhuy and I discussed your suggestion to "specify distributions by a finite number of parameters" and were wondering what the use-case of this could be? Do I understand you correctly that in essence, it wouldn't change henhuy suggestion other than allowing more values of type string other than fixed/discrete/continuous, for example Gaussian. I suppose in that case somewhere would need to be defined in which order mean, variance, etc. are given.

[jnnr]

Hi @chrwm!
The usecase would be the similar to the usecase of that feature in general: Support creation of several concrete scenarios by specifing some higher-level parameters. This can be applied in scenario comparison and sensitivity analysis.

You are right, this is an extension of @henhuy s proposal. In fact, I wonder what "discrete" or "continous" should mean without defining some distribution to sample from?

To make it a bit more concrete:

• fixed: pass a single value
• list: pass a list of values
• continous: pass parameters of a distribution (in the correct order), e.g.
  • uniform: lower_bound, upper_bound
  • gaussian: mean, variance
  • etc.

update replaced "discrete" with "list", as you could think of discrete distributions as well. List would imply that all values are equally probable to be drawn.

[chrwm]

The bandwidth_types will be based on NetCDF conventions and extended with custom conventions from the SEDOS project. Your proposals and definitions can be included as the latter.

[chrwm]

@chrwm from 2022-07-28 AP1 meeting:

• in the case of performance issues due to many timeseries per csv-table consider ANSI-encoding instead of UTF-8
• then make sure to address encoding errors script-based