Merge develop branch into main branch #87
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add MLStats related classes. * Add test stats test cases * Change line/branch test coverage to 0.7 * Add stats TransportAction * Register MLStatsNodesTransportAction to Plugin Add jacocoExclusions to build.gradle * Add lombok support Add test cases to improve ut coverage * Change MLStat.clusterLevel from Boolean to boolean
* Add RestStatsMLAction and UT * Register Stats REST API in MachineLearningPlugin Decrease line coverage from 0.8 to 0.7 due to this change * Add todo to remind adding more logic to triage stats requests based on node type
Add pull request templates
* Add issue templates Add pull request templates * Change REST API path from opendistro to opensearch
* add client lib predict and train interfaces * Add exception check for UT cases * fix exception check for UT cases in train method
Add interface MLAlgo and ModelSerDeSer.
* add KMeans train and tribuo utils
* add dependencies apache, oracle tribuo
* update the rfc doc * add instructions for client use
* KMeans predict
* Update ml-rfc.md * Update ml-rfc.md
* Add MLTask data model. * Align task type * Modify MLTaskTest case * Add error field to store failure message Adjust taskType and state to use Enum type. * Move @Getter to class level
* Add linear regression algorithm with train function.
* Add MLTask data model. * Align task type * Modify MLTaskTest case * Add error field to store failure message Adjust taskType and state to use Enum type. * Move @Getter to class level * Add MLNodeUtils to check if node is ML node Add MLIndicesHandler to handle persist model * Modify license header * Remove unused variable
* Add MLTask data model. * Align task type * Modify MLTaskTest case * Add error field to store failure message Adjust taskType and state to use Enum type. * Move @Getter to class level * Add MLNodeUtils to check if node is ML node Add MLIndicesHandler to handle persist model * Modify license header * Remove unused variable * Add MLTaskManager and test cases * Add javadoc for MLTaskManager Add todo to capture future task: utilize Circuit Breaker to limit adding new tasks.
* add MLEngine interface
* Enable fgac support for ml plugin. Signed-off-by: Alex <pengsun@amazon.com> * Update opensearch and dependency version to 1.0.0 Signed-off-by: Alex <pengsun@amazon.com> * Enable the integration test in the CI workflow and gradlew build Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
Signed-off-by: Alex Sun <pengsun@dev-dsk-pengsun-2c-c6fbcf50.us-west-2.amazon.com> Co-authored-by: Alex Sun <pengsun@dev-dsk-pengsun-2c-c6fbcf50.us-west-2.amazon.com>
…est (#60) accordingly. Signed-off-by: Alex Sun <pengsun@dev-dsk-pengsun-2c-c6fbcf50.us-west-2.amazon.com> Co-authored-by: Alex Sun <pengsun@dev-dsk-pengsun-2c-c6fbcf50.us-west-2.amazon.com>
* Fix an serialize issue of Node Stats API and update the integration test accordingly. Signed-off-by: Alex Sun <pengsun@dev-dsk-pengsun-2c-c6fbcf50.us-west-2.amazon.com> * Update the CI workflow. Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex Sun <pengsun@dev-dsk-pengsun-2c-c6fbcf50.us-west-2.amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
Signed-off-by: Alex <pengsun@amazon.com>
is running very slow. Signed-off-by: Alex <pengsun@amazon.com>
same name. Signed-off-by: Alex <pengsun@amazon.com>
Signed-off-by: Alex <pengsun@amazon.com>
* Update the ml index name. Signed-off-by: Alex <pengsun@amazon.com> * Update the integration test max waiting time since the CI workflow host is running very slow. Signed-off-by: Alex <pengsun@amazon.com> * Update the ml index name to be more unique to avoid customer using the same name. Signed-off-by: Alex <pengsun@amazon.com> * Update the ml index name to avoid using os Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
Signed-off-by: Alex <pengsun@amazon.com>
./gradlew release Signed-off-by: Alex <pengsun@amazon.com>
Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
…id the conflict with search XContent input. (#71) * Create JvmService instance on demand. Signed-off-by: Alex <pengsun@amazon.com> * Move the ml_parameters from XContent to the request parameters to avoid the conflict with search XContent input. Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
* Create JvmService instance on demand. Signed-off-by: Alex <pengsun@amazon.com> * Move the ml_parameters from XContent to the request parameters to avoid the conflict with search XContent input. Signed-off-by: Alex <pengsun@amazon.com> * Fix the security risks found by PenTest. 1. unhandled 500 server error. 2. Insecure Deserialization Signed-off-by: Alex <pengsun@amazon.com> * Remove unnecessory '*' from the welcome list of model deserializer. Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
* Create JvmService instance on demand. Signed-off-by: Alex <pengsun@amazon.com> * Move the ml_parameters from XContent to the request parameters to avoid the conflict with search XContent input. Signed-off-by: Alex <pengsun@amazon.com> * Fix the security risks found by PenTest. 1. unhandled 500 server error. 2. Insecure Deserialization Signed-off-by: Alex <pengsun@amazon.com> * Remove unnecessory '*' from the welcome list of model deserializer. Signed-off-by: Alex <pengsun@amazon.com> * Update readme to add more information. Signed-off-by: Alex <pengsun@amazon.com> * Add developer guide to the document. Signed-off-by: Alex <pengsun@amazon.com> * Add documents for ml-commens. Signed-off-by: Alex <pengsun@amazon.com> * Sync the build scripts Signed-off-by: Alex <pengsun@amazon.com> * Remove the dependencies added to support Mleap. Signed-off-by: Alex <pengsun@amazon.com> Co-authored-by: Alex <pengsun@amazon.com>
Signed-off-by: Jackie Han <hnyng@amazon.com>
Signed-off-by: Jackie Han <hnyng@amazon.com>
Signed-off-by: Jackie Han <hnyng@amazon.com>
Signed-off-by: Jackie Han <hnyng@amazon.com>
ylwu-amzn
approved these changes
Nov 16, 2021
amitgalitz
approved these changes
Nov 16, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.