Skip to content

[ML] model plots get overwritten with actual==0 in outage scenario #180

Open
@hendrikmuhs

Description

@hendrikmuhs

In the event of an outage in the ML node there is a skew between the time of the model snapshot and the time of the datafeed/data counts.

When the job gets started up again the datafeed starts feeding data starting from time x, while the model is at time y, and y < x. The difference between y and x can be worst case 3 hours.

The datafeed sends a skipTime control message to notify autodetect about the skew, notifying that it starts data from time x instead of y.

Still, for the time window [y, x] autodetect writes model plots, usually 'zero' results (actual == 0), this is technically correct but has side-effects: On the plugin side the empty results can overwrite already existing model plots written before the outage with actuals !=0.

Therefore we suspect a bug in skipTime being not effective for model plots.

Needs repro+debug.

Related issue: elastic/elasticsearch#31867

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions