Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check state before deleting model or task #725

Merged
merged 3 commits into from
Feb 21, 2023

Conversation

rbhavna
Copy link
Collaborator

@rbhavna rbhavna commented Feb 7, 2023

Signed-off-by: Bhavana Goud Ramaram rbhavna@amazon.com

Description

This PR check the condition for model state before deleting a model. If the model is in loaded/loading/partially loaded state, it throws exception and tells user to unload the model first. Previously, the delete API would delete the model irrespective of any model state which was no desirable.

Also checks condition for task state before deleting a task. If task is in running state, it throws an exception.

Issues Resolved

#665

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Bhavana Goud Ramaram <rbhavna@amazon.com>
@rbhavna rbhavna requested a review from a team February 7, 2023 17:05
Copy link
Collaborator

@Zhangxunmt Zhangxunmt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. But to be on the safe side, are you still planning to check the model and task status in the cache memory to make sure they are not deleted? You can add it in a separate PR. https://github.com/opensearch-project/ml-commons/blob/main/plugin/src/main/java/org/opensearch/ml/model/MLModelCache.java, and, https://github.com/opensearch-project/ml-commons/blob/main/plugin/src/main/java/org/opensearch/ml/task/MLTaskCache.java.

@rbhavna
Copy link
Collaborator Author

rbhavna commented Feb 7, 2023

LGTM. But to be on the safe side, are you still planning to check the model and task status in the cache memory to make sure they are not deleted? You can add it in a separate PR. https://github.com/opensearch-project/ml-commons/blob/main/plugin/src/main/java/org/opensearch/ml/model/MLModelCache.java, and, https://github.com/opensearch-project/ml-commons/blob/main/plugin/src/main/java/org/opensearch/ml/task/MLTaskCache.java.

Yes, I think we might have to do it.

}
});
}, e -> { actionListener.onFailure(new MLResourceNotFoundException("Fail to find model")); }), () -> context.restore()));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From line 81, we are going to restore context before running listener to delete model. Will it fail on security enabled cluster?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested it on security enabled cluster and it works. Adding test results here.
Screenshot 2023-02-20 at 1 13 16 PM

@rbhavna rbhavna merged commit 7ae4192 into opensearch-project:2.x Feb 21, 2023
ylwu-amzn pushed a commit to ylwu-amzn/ml-commons that referenced this pull request Mar 3, 2023
Signed-off-by: Bhavana Goud Ramaram <rbhavna@amazon.com>
ylwu-amzn pushed a commit to ylwu-amzn/ml-commons that referenced this pull request Mar 4, 2023
Signed-off-by: Bhavana Goud Ramaram <rbhavna@amazon.com>
ylwu-amzn pushed a commit to ylwu-amzn/ml-commons that referenced this pull request Mar 4, 2023
Signed-off-by: Bhavana Goud Ramaram <rbhavna@amazon.com>
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ylwu-amzn added a commit that referenced this pull request Mar 4, 2023
…783)

* check state before deleting model or task (#725)

Signed-off-by: Bhavana Goud Ramaram <rbhavna@amazon.com>
Signed-off-by: Yaliang Wu <ylwu@amazon.com>

* fix breaking change

Signed-off-by: Yaliang Wu <ylwu@amazon.com>

---------

Signed-off-by: Bhavana Goud Ramaram <rbhavna@amazon.com>
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
Co-authored-by: Bhavana Ramaram <rbhavna@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants