2.9.0.0

Version 2.9.0.0 Release Notes

Compatible with OpenSearch 2.9.0

Features

• remote inference: add connector; fine tune ML model and tensor class (#1051)
• remote inference: add connector executor (#1052)
• connector transport actions, requests and responses (#1053)
• refactor predictable: add method to check if model is ready (#1057)
• Add basic connector access control classes (#1055)
• connector transport actions and disable native memory CB (#1056)
• restful connector actions and UT (#1065)
• Change connector access control creation allow empty list (#1069)

Enhancements

• create model group automatically with first model version (#1063)
• init master key automatically (#1075))

Infrastructure

• Adding an integration test for redeploying a model (#1016)
• add unit test for connector class in commons (#1058)
• remote inference: add unit test for model and register model input (#1059)
• remote inference: add unit test for StringUtils and remote inference input (#1061)
• more UT for rest and trasport actions (#1066)
• remote inference: add unit test for create connector request/response (#1067)
• Add more UT for remote inference classes (#1077)
• IT Security Tests for model access control (#1095)
• Add integration tests for the remote inference feature (#1073)
• Temporarily ignore OpenAI Tests (#1118)
• Increase sleep time on remote inference integration tests (#1124)
• fix remote inference it: update job interval (#1129)
• Add null key check for remote inference integration tests (#1113)
• Reenable OpenAI Remote Inference Tests (#1127)

Bug Fixes

• Add missing codes from pen test fix (#1060)
• fix cannot specify model access control parameters error (#1068)
• fix memory circuit breaker (#1072)
• PenTest fixes: error codes and update model group fix (#1074)
• Fix rare private ip address bypass SSRF issue (#1070)
• leftover in the 404 Not Found return error (#1079)
• modify error message when model group not unique is provided (#1078)
• stash context before accessing ml config index (#1092)
• fix init master key bug (#1094)
• delete connector successfully if model index is missing (#1106)
• fix encoding US-ASCII error (#1126)

Documentation

• model access control documentation (#966)
• updating docs for model group id (#980)

Maintenance

• Increment version to 2.9.0-SNAPSHOT (#955)
• Manual CVE backport (#1008)
• Fix build. (#1018)
• Fix the refactor change brought by core backport (#1047)
• change to compileOnly to avoid jarhell (#1062)
• Update ml CI workflow checkout ref to be PR commit (#1114)
• Add shell bash settings for windows github actions to export credentials (#1116)

2.8.0.0

Version 2.8.0.0 Release Notes

Compatible with OpenSearch 2.8.0

Experimental Features

• Model access control. (#928)

Enhancements

• Add a setting to enable/disable model url in register API (#871)
• Add a setting to enable/disable local upload while registering model (#873)
• Check hash value for the pretrained models (#878)
• Add pre-trained model list (#883)
• Add content hash value for the correlation model. (#885)
• Set default access_control_enabled setting to false (#935)
• Enable model access control in secure reset IT (#940)
• Add model group rest ITs (#942)

Bug Fixes

• Fix class not found exception when deserialize model (#899)
• Fix publish shadow publication dependency issue (#919)
• Fix model group index not existing model version query issue and SecureMLRestIT failure ITs (#933)
• Fix model access mode upper case bug (#937)

Documentation

Maintenance

• Increment version to 2.8.0-SNAPSHOT (#896)

Refactoring

• Change mem_size_estimation to memory_size_estimation (#868)

2.7.0.0

Version 2.7.0.0 Release Notes

Compatible with OpenSearch 2.7.0

Experimental Features

• Add metrics correlation algorithm. (#845)

Enhancements

• Add model auto deploy feature (#852)
• Add memory consumption estimation for models in profile API (#853)
• Add text docs ML input (#830)
• Add allow custom deployment plan setting; add deploy to all nodes field in model index (#818)
• Add exclude nodes setting. (#813)
• set model state as partially loaded if unload model from partial nodes (#806)

Bug Fixes

• change to old method to fix missing method createParentDirectories (#759)
• fix delete model API (#861)
• fix breaking changes of Xcontent namespace change (#838)
• Change the ziputil dependency to fix a potential security concern (#824)
• fix checkstyle version (#792)
• Typo fix and minor improvement in maven-publish GHA workflow (#757)

Documentation

• add docker-compose file for starting cluster with dedicated ML node (#799)

Maintenance

• Increment version to 2.7.0-SNAPSHOT (#742)
• Publish snapshots to maven via GHA (#754)

Refactoring

• rename API/function/variable names (#822)
• rename model meta/chunk API (#827)

2.6.0.0

Compatible with OpenSearch 2.6.0.

Experimental Features

• enable prebuilt model. (#729)

Enhancements

• tune error message for no ML node case (#699)
• [experimental] Enhance profile API to add model centric result controlled by view parameter (#714)
• [experimental] tune model config: change pooling mode to optional (#724)
• check state before deleting model or task (#725)

Bug Fixes

• fixing gradle issue (#735)
• fix delete model API (#748)

Documentation

• Update gpu doc with docker test (#702)
• add text embedding API example doc (#710)
• fix profile API in example doc (#712)
• change model url to public repo in text embedding model example doc (#713)
• json listing of all the pretrianed models (#730)

Maintenance

• Increment version to 2.6.0-SNAPSHOT (#671)

Refactoring

• add DL model class (#722)

2.5.0.0

Compatible with OpenSearch 2.5.0

Features

• Add native memory circuit breaker. (#689)

Enhancements

• add more parameters for text embedding model (#640)
• add more pooling method and refactor (#672)
• add ML task timeout setting and clean up expired tasks from cache (#662)
• change only run on ml node setting default value to true (#686)

Infrastructure

• unit tests coverage for load/unload/syncup (#592)
• Add .whitesource configuration file (#626)
• bump djl to 0.20 and add onnxruntime-gpu dependency (#644)
• Remove jackson-databind and jackson-annotations dependencies now coming from core (#652)
• Revert "Remove jackson-databind and jackson-annotations dependencies now coming from core" (#687)
• Adding backwards compatibility test for ml-commons plugin (#681)
• Change the inheritance of the BWC test file (#692)

Documentation

• Updated MAINTAINERS.md format (#668)
• Updating maintainers list (#663)
• add doc about how to setup GPU ML node (#677)

Maintenance

• Increment version to 2.5.0-SNAPSHOT (#513)

Refactoring

• change task worker node to list; add target worker node to cache (#656)

2.0.0.0

Compatible with OpenSearch 2.0.0

Enhancements

• Add circuit breaker trigger count stat.(#274)

Bug Fixes

• support dispatching execute task; don't dispatch ML task again (#279)
• Fix NPE in anomaly localization (#280)
• create model/task index with correct mapping (#284)

Infrastructure

• drop support for JDK 14 (#267)
• Add UT/IT Coverage for action/models and action/tasks. (#268)
• Default qualifier to alpha1 and fix workflows (#269)
• Remove additional vars in build.gradle that are not used (#271)
• Add UT for Search transport action. (#272)
• updated issue templates for bugs and features. (#273)
• Add more test to improve coverage of abstract search action(#275)
• Add UT for RestMLExecuteAction, and remove it out from the jacoco exclusive list. (#278)
• add coverage badges (#281)
• Re-enable docker image tests for 2.0. (#288)

Maintenance

• Change 2.0-alpha1 to 2.0-rc1. (#282)
• bump RCF version to 3.0-rc2.1 (#289)
• bump tribuo version to 4.2.1 (#312)

Refactoring

• Removed RCF jars and updated to fetch RCF 3.0-rc2 from maven (#277)

1.3.0.0

Compatible with OpenSearch 1.3.0

Features

• add anomaly localization implementation (#103)
• refactor ML task data model; add create ML task index method (#116)
• integration step 1 and 2 for anomaly localization (#113)
• anomaly localization integration step 3 (#114)
• support train ML model in either sync or async way (#124)
• anomaly localization integration step 4 and 5 (#125)
• add train and predict API (#126)
• add ML Model get API (#117)
• integrate tribuo anomaly detection based on libSVM (#96)
• add ML Delete model API (#136)
• add fixed in time rcf (#138)
• Add ML Model Search API (#140)
• add circuit breaker (#142)
• add batch RCF for non-time-series data (#145)
• Add ML Task GET/Delete API (#146)
• Add Search Task API and Refactor search actions and handlers (#149)
• add minimum top contributor candidate queue size (#151)
• add more stats: request/failure/model count on algo/action level (#159)
• add tasks API in Client (#200)

Enhancements

• support float type in data frame (#129)
• support short and long type in data frame (#131)
• use threadpool in execute task runner (#156)
• do not return model and task id in response (#171)
• more strict check on input parameters by applying non-coerce mode (#173)
• move anomaly localization to the last position to avoid BWC issue (#189)

Bug Fixes

• use latest version of tribuo to fix modify thread group permission issue (#112)
• fix jarhell error from SQL build (#137)
• fix EpochMilli parse error in MLTask (#147)
• fix permission when accessing ML system indices (#148)
• fix system index permission issue in train/predict runner (#150)
• cleanup task cache once task done (#152)
• fix update task semaphore; don't return task id for sync request (#153)
• restore context after accessing system index to check user permission on non-system index (#154)
• fix verbose error message thrown by invalid enum (#167)
• fix no permission to create model/task index bug;add security IT for train/predict API (#177)

Infrastructure

• add git ignore file (#92)
• change common utils to 1.2 snapshot;add more test (#94)
• Remove jcenter dependency (#121)
• add integration test for train and predict API (#157)
• fix build/CI and add backport workflow (#161)
• publish ml client to maven (#165)
• Add integ tests for model APIs (#166)
• add security IT (#168)
• fix maven group (#170)
• add more UT for ml-algorithms (#182)
• add java 8 to CI workflow (#194)
• add more UT and IT for rest actions (#192)
• add more UT to client module (#203)
• add more UT for task manager/runner (#206)
• create config and workflow files for release note (#209)
• use 1.3.0 docker to run CI (#212)

Documentation

• Add support for codeowners to repo (#91)
• add how to develop new function doc to readme (#95)
• update license header (#134)

Maintenance

• Bump version to 1.2 (#90)
• bump to 1.2.3 (#110)
• bump to 1.3.0 (#115)

Refactoring

• Merge develop branch into main branch (#87)
• refactor API input/output/URL; add execute API for non-model based algorithm (#93)
• cleanup code and refactor (#106)
• support registering ML objects; refactor ML engine interface (#108)
• refactor persisting ML model (#109)
• refactor transport APIs;fix class cast exception (#127)
• add ML custom exceptions (#133)
• rename tribuo AD algorithm name (#144)