[FIX] Aggregate unbalanced datasets

[AzulGarza]

This PR fixes #189.
Tests were added comparing the fixes with the deprecated function aggregate_before (which works as expected in this case).

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[NudnikShpilkis]

@FedericoGarza looks like the change to aggregate works, but reconcile still fails. Following my example from earlier

train_df = hier_df.query("ds <= @pd.to_datetime('2019-12-31')")
test_df = hier_df.query("ds > @pd.to_datetime('2019-12-31')")

fcst = StatsForecast(
    models=[
        sfm.AutoARIMA(season_length=12, alias='ARIMA'),
    ],
    freq='M',
    n_jobs=-1,
)

hrec = HierarchicalReconciliation(
    reconcilers=[
        hfm.BottomUp(),
        hfm.MinTrace(method='mint_shrink'),
    ]
)

fcst.fit(df=train_df)
fcst_df = fcst.forecast(h=12, fitted=True)
fitted_df = fcst.forecast_fitted_values()

fcst_df = hrec.reconcile(
    Y_hat_df=fcst_df,
    Y_df=fitted_df,
    S=S_df,
    tags=tags,
)

We get ValueError: cannot reshape array of size 64 into shape (6,newaxis) becausefitted_df have a length of 64, while S_df has a length of 6. You'll need to rewrite reconcile to address the reshaping.

[NudnikShpilkis]

Not a bug per se, but I'd clarify the exception for not including Y_df in reconcile. The type hints imply it's always optional, but it's only optional for bootstrap or permbu methods. So the exception "you need to pass Y_df" is confusing in light of the type hint.

[candalfigomoro]

My current workaround is to create a copy of train_df without the leading zeros and use that for the forecasting model. For reconcile() I still use the version with the leading zeros.

Note that aggregate() also creates all-zero time series for missing hierarchical combinations, so you also have to add all-zero forecasts to Y_hat_df/fcst_df for unique_ids artificially created by aggregate().

[AzulGarza]

hey @NudnikShpilkis and @candalfigomoro! Thank you for your feedback! I've updated the branch to consider this case in the reconcile method. Could you try uninstalling and installing the library again from this branch? :)

Thank you!

[NudnikShpilkis]

@FedericoGarza looks like the code now runs without error. I haven't tested for validity of predictions though.

[NudnikShpilkis]

@candalfigomoro

My current workaround is to create a copy of train_df without the leading zeros and use that for the forecasting model. For reconcile() I still use the version with the leading zeros.

What's the exact formulation of your code? If you fit on the data without the leading zeros and then run forecast on data with the leading zeros it refits, doesn't it?
Where are you passing the data with the leading zeros?