Currently SCD Type 2 supports only restating the model from the beginning of time. One could also restate a model at any point in history too but also end with the latest date. This how how it would work:
Lets say I am restating Jan 10th
- Delete from the table where all valid_from >= Jan 10th
- Update the table where all valid_to >= Jan 10th
This would reset the table to look like it was last run before the given start date.
Ideally, once this is implemented, when we create a preview clone of a model we would then configure the clone to restate this latest model with the new logic.
Suggestion for resolving that is to update the SCD Type 2 logic to always go through this delete/update logic on each insert. That way in most cases where we are just processing the newest interval it would be no-op (does require a scan though) and when we have to restate it would delete/update as expected.
Currently SCD Type 2 supports only restating the model from the beginning of time. One could also restate a model at any point in history too but also end with the latest date. This how how it would work:
Lets say I am restating Jan 10th
This would reset the table to look like it was last run before the given start date.
Ideally, once this is implemented, when we create a preview clone of a model we would then configure the clone to restate this latest model with the new logic.
Suggestion for resolving that is to update the SCD Type 2 logic to always go through this delete/update logic on each insert. That way in most cases where we are just processing the newest interval it would be no-op (does require a scan though) and when we have to restate it would delete/update as expected.