Skip to content

[ML] Recalculate memory usage before upgrading model state #1585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

droberts195
Copy link
Contributor

When a persist control message with arguments is received
by the anomaly detector it doesn't go through the standard
chain of persistence calls, as it unconditionally rewrites
the state (even if no data has been seen) and includes only
the anomaly detector state rather than the categorizer state
too. Because of this the memory usage was not being
recalculated prior to persisting the state as would normally
happen. This PR rectifies that omission.

Fixes one of the problems detailed in
elastic/elasticsearch#64665 (review)

When a persist control message with arguments is received
by the anomaly detector it doesn't go through the standard
chain of persistence calls, as it unconditionally rewrites
the state (even if no data has been seen) and includes only
the anomaly detector state rather than the categorizer state
too.  Because of this the memory usage was not being
recalculated prior to persisting the state as would normally
happen.  This PR rectifies that omission.

Fixes one of the problems detailed in
elastic/elasticsearch#64665 (review)
@droberts195
Copy link
Contributor Author

I've labelled this >non-issue because the fixed functionality is logically part of elastic/elasticsearch#64665, and that's already going to be release noted for 7.11.

Copy link
Contributor

@tveasey tveasey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@droberts195 droberts195 merged commit f98e9f1 into elastic:master Nov 18, 2020
@droberts195 droberts195 deleted the refresh_memory_before_state_upgrade branch November 18, 2020 10:53
droberts195 added a commit to droberts195/ml-cpp that referenced this pull request Nov 18, 2020
When a persist control message with arguments is received
by the anomaly detector it doesn't go through the standard
chain of persistence calls, as it unconditionally rewrites
the state (even if no data has been seen) and includes only
the anomaly detector state rather than the categorizer state
too.  Because of this the memory usage was not being
recalculated prior to persisting the state as would normally
happen.  This PR rectifies that omission.

Fixes one of the problems detailed in
elastic/elasticsearch#64665 (review)

Backport of elastic#1585
droberts195 added a commit that referenced this pull request Nov 18, 2020
When a persist control message with arguments is received
by the anomaly detector it doesn't go through the standard
chain of persistence calls, as it unconditionally rewrites
the state (even if no data has been seen) and includes only
the anomaly detector state rather than the categorizer state
too.  Because of this the memory usage was not being
recalculated prior to persisting the state as would normally
happen.  This PR rectifies that omission.

Fixes one of the problems detailed in
elastic/elasticsearch#64665 (review)

Backport of #1585
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants