Summary
An online filter update can receive a new vintage that revises already-assimilated values without adding any new reference-date slots. This is a valid revision-only update: the number of appended observations is zero, not an error.
The current length guard requires the new history to be strictly longer (T > n_processed). It should allow equal lengths (T >= n_processed) so that revision-only updates can follow the configured revision policy.
Minimal example
Suppose the filter has already assimilated:
dates = [d1, d2]
values = [10.0, 11.0]
The next data vintage contains:
dates = [d1, d2]
values = [10.5, 11.0]
There are no newly appended observations, but the prefix contains a revision. When replay-on-revision is disabled, the intended behavior is to ignore the revised prefix, assimilate an empty suffix, and continue to the forecast (or fork over the provisional tail). A strict growth check instead rejects the vintage because T == n_processed.
We encountered this in a daily NSSP particle-filter backtest: NY and US smoke runs failed at revision-only origins even though their histories had not shrunk or changed structurally.
Expected behavior
T > n_processed: assimilate the appended suffix according to the normal append-only path.T == n_processed, with the same ordered dates: treat the appended suffix as empty and apply the configured revision policy (no-op when revisions are ignored; full replay when replay is enabled).T < n_processed: continue to reject this as a shortened history unless a full replay policy explicitly handles it.- Reordered or changed prefix dates should continue to be treated as structural changes.
Suggested regression test
Add a two-slot assimilated history followed by a same-length vintage with one revised value. Assert that the update range is empty and that the operation does not error when replay-on-revision is disabled. Retain separate tests for genuinely shortened and reordered histories.
The downstream guard was corrected from T > n_processed to T >= n_processed; the important distinction is between a zero-length appended suffix and a negative/shrinking one.
Summary
An online filter update can receive a new vintage that revises already-assimilated values without adding any new reference-date slots. This is a valid revision-only update: the number of appended observations is zero, not an error.
The current length guard requires the new history to be strictly longer (
T > n_processed). It should allow equal lengths (T >= n_processed) so that revision-only updates can follow the configured revision policy.Minimal example
Suppose the filter has already assimilated:
The next data vintage contains:
There are no newly appended observations, but the prefix contains a revision. When replay-on-revision is disabled, the intended behavior is to ignore the revised prefix, assimilate an empty suffix, and continue to the forecast (or fork over the provisional tail). A strict growth check instead rejects the vintage because
T == n_processed.We encountered this in a daily NSSP particle-filter backtest: NY and US smoke runs failed at revision-only origins even though their histories had not shrunk or changed structurally.
Expected behavior
T > n_processed: assimilate the appended suffix according to the normal append-only path.T == n_processed, with the same ordered dates: treat the appended suffix as empty and apply the configured revision policy (no-op when revisions are ignored; full replay when replay is enabled).T < n_processed: continue to reject this as a shortened history unless a full replay policy explicitly handles it.Suggested regression test
Add a two-slot assimilated history followed by a same-length vintage with one revised value. Assert that the update range is empty and that the operation does not error when replay-on-revision is disabled. Retain separate tests for genuinely shortened and reordered histories.
The downstream guard was corrected from
T > n_processedtoT >= n_processed; the important distinction is between a zero-length appended suffix and a negative/shrinking one.