multi_models = True and output_chunk_length > 1 : index future covariates relatively to the forecasted step

[swertz]

Is your feature request related to a current problem? Please describe.
When using a regression model with multi_models = True, output_chunk_length > 1, and future covariates, each model sees all future covariates specified as the future covariates are index with respect to the "absolute" first point of the forecasted output chunk, rather than relative to the point being forecasted by a specific model.

For long output chunks, this can result in the models being fed a large number of highly correlated features, decreasing their performance. For instance with output_chunk_length=96 anbd lags_future_covariates=[0], each model sees every feature 96 times, while I think it would make more sense, in general, for each model to only see the covariates at the moment it is trying to forecast.

Describe proposed solution
Providing the possibility to specify the future covariate lags with respect to the forecasted points in the output chunk, rather than with respect to the first point in the chunk.

Describe potential alternatives
Alternatives were discussed in the issues linked below (such as the "macking" approach, or using the forecasting horizon as a feature).

Additional context

This was already discussed in those two issues: #2650 , #2234 , but they were closed without a feature request being kept open for tracking.

I would also like to clarify something mentioned in #2234: @madtoinou wrote

When multi_models=False and the underlying model does not support multi-output prediction, ...

Does that mean that a model that supports multi-output regression, using multi_models=False in conjunction with output_chunk_length > 1 will result in a multi-output regression over the output chunk length, rather than a single model being used auto-regressively?

[dennisbader]

Hi @swertz and thanks for raising this issue.

Just to clarify this point:

For long output chunks, this can result in the models being fed a large number of highly correlated features, decreasing their performance. For instance with output_chunk_length=96 and lags_future_covariates=[0], each model sees every feature 96 times, while I think it would make more sense, in general, for each model to only see the covariates at the moment it is trying to forecast.

The case you describe would actually only add every feature a single time (lags_future_covariates=[0]). The model will only see every feature 96 times for the cases lags_future_covariates=(0,96) or lags_future_covariates=[i for i in range(96)]

I agree that allowing the future covariates to be extracted relative to the predicted step is valuable.

We're also currently working the "DirRec" / Direct Recursive SKLearnModel model, which would also benefit from that.

Before we start implementing I would like to get a clear picture of what we actually want to add support for.

• I believe, we could add a new parameter to the model constructor that enables/disables this "shifting" behavior for future covariate lags.
  • Should this shifting behavior be component-specific or enforced for all future covariate components?
• Should the user be able to still use multiple future covariate lags or only the lag matching the output_chunk_length?
• What should we do with historic future covariates lags? (e.g. lags<0)
• We need to think of the value convention for this new mode. Should we keep our existing convention (e.g. lags_future_covariates=[95] for output_chunk_length=96 or lags_future_covariates=[0]). I'm a bit worried of confusing people even more with introducing new conventions.

[swertz]

Hi @dennisbader , thanks for your feedback!

The case you describe would actually only add every feature a single time (lags_future_covariates=[0]). The model will only see every feature 96 times for the cases lags_future_covariates=(0,96) or lags_future_covariates=[i for i in range(96)]

Yes you are correct, that was a typo!

We're also currently working the "DirRec" / Direct Recursive SKLearnModel model, which would also benefit from that.

Nice!

Should this shifting behavior be component-specific or enforced for all future covariate components?

I suppose it could be component-specific, just like the lags already are.

Should the user be able to still use multiple future covariate lags or only the lag matching the output_chunk_length?

It could make sense to keep the same format as for the current future covariate lags specification... One could then e.g. use the values "one step before and one step later" relative to every step being forecasted.

The component-specific "relative" flag would indicate whether those lags are "relative" or "absolute". It's true that it makes for a complex API... This would give e.g.:

output_chunk_length=96,
lags_future_covariates = {
    "comp1": (-1, 1),
    "comp2": [0, 95],
},
lags_future_covariates_relative = {
    "comp1": True,
    "comp2": False  # default if not specified (current behaviour)
}

What should we do with historic future covariates lags? (e.g. lags<0)

Is there a reason for treating them differently from the "true" future lags?

We need to think of the value convention for this new mode. Should we keep our existing convention (e.g. lags_future_covariates=[95] for output_chunk_length=96 or lags_future_covariates=[0]).

But currently, 95 is relative to the first point being forecasted, right?

For instance, for an output_chunk_length of 4:

timestamp	TS lag	FC lag (rel. to 1st step)	FC lag (rel. to current step)
9h00	0	0	0
9h15	1	1	0
9h30	2	2	0
9h45	3	3	0

If I want the model to only use the future covariate at the same timestamp as the step being forecasted, it would seem more logical to specify it as lags_future_covariates=[0].

I don't quite see why lags_future_covariates=[4] (here) would make sense in this context... In the existing convention, this means you'd be using only the covariate value at 9h45 to forecast for all steps (9h, 9h15, ...). But if you want to specify that the model uses only the covariate value at time X to forecast at time X, specifying lags_future_covariates[4] is not intuitive to me.

[dennisbader]

Thanks for the response @swertz :)

What should we do with historic future covariates lags? (e.g. lags<0)

Is there a reason for treating them differently from the "true" future lags?

Sounds good. So lags_future_covariate=[-1] (let's call it lags_fc from now on) would correspond to "Only use fc at time t-1 when predicting t)?

We need to think of the value convention for this new mode. Should we keep our existing convention (e.g. lags_future_covariates=[95] for output_chunk_length=96 or lags_future_covariates=[0]).

But currently, 95 is relative to the first point being forecasted, right?

Yes, that's true. I think your proposal makes more sense. I just thought about the added complexity to the lags handling from a backend perspective (e.g. extreme lags handling for determining historical forecast start / end points, input time frame validation for training / prediction, etc.). We'll just have to thoroughly unit test it 👍

lags_future_covariates_relative = {
    "comp1": True,
    "comp2": False  # default if not specified (current behaviour)
}

Adding a flag for activating this new behavior (component-specific or global) sounds good. We'll need to find a better name at some point, as the current logic is already "relative" to forecast start. But there is time for that :) The definition could look something like below.

some_flag_name: Union[bool, dict[str, bool]] = False

A bool would be globally applied, and dict[str, bool] will be component-specific (including the default_lags key that would set it for the remainder of columns not specifically set. E.g.

some_flag_name = False  # use current behavior for all components
some_flag_name = True  # use new behavior for all components
some_flag_name = {"a": True, "default_lags": False}  # use new behavior only for component "a"

How does that sound?

[pfwnicks]

Awesome that this can become a feature! Looking forward to reducing the dimensionality of some of our models.

[swertz]

Sounds good. So lags_future_covariate=[-1] (let's call it lags_fc from now on) would correspond to "Only use fc at time t-1 when predicting t)?

That's what makes the most sense to me!

I just thought about the added complexity to the lags handling from a backend perspective (e.g. extreme lags handling for determining historical forecast start / end points, input time frame validation for training / prediction, etc.).

Of course, I was not thinking in those terms :)

How does that sound?

That sounds perfect! 👍