Possibly disable month and year for Duration parameter type

[jkiviluo]

While Python, Julia and SpineOpt all handle month and year correctly for Duration parameter (in Python using dateutils.relativedelta), there are some drawbacks:

• months and years have changing duration --> most models don't know how to deal with those (user could use them and get weird behaviour)
• Python's dateutils.relativedelta (and probably Julia's equivalent) is slower than datetime.timedelta, since the latter does not deal with complex if clauses

Our options:

• keep doing as now (use relativedelta, leave it to user not to srew-up), but:
  • improve documentation
  • add a warning note where appropriate
• implement timedelta as the default parameter type for Duration (leave relativedelta as an option)
  • speed improvement should be tested
• switch completely to timedelta
  • check if anyone is using months or years as Duration in their models - and are these important use cases where the workaround is non-satisfactory

Relates to: #319

[DillonJ]

I know this is not the intention here but just wanted to make sure that this wouldn't impact time pattern syntax where it should still be ok to use months and years which we are doing now quite extensively

[manuelma]

My preference here would be to add a clear note so users know what they're messing with - but don't disable the support. I am not sure Julia is affected by this since they don't provide any simpler API. The only API does things right so probably it's well optimised.

[jkiviluo]

Please note: this is about resolution parameter. We would still allow to set parameter values for e.g. a particular month. Setting parameters for a particular month doesn't have the same issue, since then one is talking about a particular month that has a particular number of days - instead of duration of abstract month where the number of days is not known.

[manuelma]

I think to be fully clear we should say that this is about the resolution property for the TimeSeriesFixedResolution.

@jkiviluo I should also point out that if the user gives the resolution, they also need to give the start, and that makes everything concrete.

[jkiviluo]

Oh no, I've mixed things up again. I was really thinking about the Duration parameter type. For TimeSeriesFixedResolution things are indeed clear. I've clean up the issue name and description.

[jkiviluo]

Although, if you have a DB with a temporal resolution of one month, most models will not be able to handle it correctly unless it's implemented with TimeSeriesVariableResolution.

I guess it could be a DB level choice too. I'm now working on the generic DB and there I need to worry about these things. For SpineOpt DB 1 month resolution would be fine though.

[manuelma]

I think I misled you to use the word 'resolution' - my point was that we should try and use the terminology in the docs as much as possible (https://spine-database-api.readthedocs.io/en/0.8-dev/parameter_value_format.html)

Just curious @jkiviluo when you say most models you mean which ones in particular?

[manuelma]

@jkiviluo also, is the generic DB specified somewhere? How are we working on this?

[soininen]

At least the Python implementation of TimeSeriesFixedResolution handles monthly and yearly resolution correctly so if you e.g. export the time stamps to .csv you get the dates you would expect.

[jkiviluo]

Just curious @jkiviluo when you say most models you mean which ones in particular?

I didn't check, but most models don't even handle variable time resolution. And even models like Backbone and FlexTool that can have variable time resolution wouldn't function if one tried to say that they should roll forward a 1 month Duration. Or if you tried to set their time Resolution parameter to 1 month, it wouldn't work either - it could be made to work, but it would require quite a bit of coding since their modelling languages don't know how to interpret 1 month (interpreting 4 weeks is trivial in comparison).

[jkiviluo]

At least the Python implementation of TimeSeriesFixedResolution handles monthly and yearly resolution correctly so if you e.g. export the time stamps to .csv you get the dates you would expect.

Right, doing a script to handle the time series on Python side would make things easier for the Resolution parameter (for models that can handle variable time resolution). However, Duration (e.g. roll forward) requires understanding of the date. While it's plausible to manage that also in Python, it can require quite a bit of rewriting on the model side depending on the particular implementation.

[jkiviluo]

Is the generic DB specified somewhere? How are we working on this?

It's currently in WP5, Task 5.2 folder. I plan to make a repository copy (as a JSON) soon. The repo would then be the place to discuss too, but for now it's just directly to me.