Allows out of band rollovers on an index without causing ISM to fail #180
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com>
Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com>
thalurur
approved these changes
Nov 2, 2021
Comment on lines
-75
to
-82
| // If we have already rolled over this index then fail as we only allow an index to be rolled over once | ||
| if (managedIndexMetaData.rolledOver == true) { | ||
| logger.warn("$indexName was already rolled over, cannot execute rollover step") | ||
| stepStatus = StepStatus.FAILED | ||
| info = mapOf("message" to getFailedDuplicateRolloverMessage(indexName)) | ||
| return this | ||
| } | ||
|
|
There was a problem hiding this comment.
If this is removed, then this rolledover attribute is not useful anymore and can be cleaned up, right?
There was a problem hiding this comment.
If this is the only place it was actively used.. then technically yes, but at the same time users will have metadata that has this boolean on it regardless so we can't remove it completely from our codebase if our parsing is strict and throws an exception when it sees an unrecognized field.
downsrob
added a commit
that referenced
this pull request
Nov 5, 2021
* Add integTest script to the repo (#94) Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Removing Usages of Action Get Call and using listeners (#100) Signed-off-by: Aditya Jindal <aditjind@amazon.com> * Enhance ISM template (#105) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Explain response still use old opendistro policy id (#109) * Explain response still use old opendistro policy id * Use hardcoded policyid setting in tests for explain response * Trying to fix flaky tests * Storing user information as part of the job when security plugin is installed (#113) Signed-off-by: Ravi Thaluru <ravi1092@gmail.com> * ISM/Notification channel support (#117) * Updates NAME of transport actions * Upgrades Kotlin version, updates dependencies on other OS plugins, adds notification plugin as a test resource and includes it in test clusters * Adds support for Channels in error notifications and notification actions * Adds support for sending notifications to channels * Adds support for publishing notifications to the legacy destinations through the Notification plugin and some cleanup * Removes notification alerting jar dependency * Adds compile only dep on commons codec for digest utils sha1 method in ism rollup * Updates Error Notification to make channel/destination nullable, and adds helper methods for publish calls * Constructs URL for legacy custom webhook Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes Feature enum and dep Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Trying something else Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Addresses comments Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Storing user object in all APIs and enabling filter of response based on user (#115) Signed-off-by: Ravi Thaluru <ravi1092@gmail.com> * Upgrade dependencies to 1.1 and build snapshot by default. (#121) Signed-off-by: dblock <dblock@amazon.com> * Security improvements (#126) Signed-off-by: Ravi Thaluru <ravi1092@gmail.com> * Removes support for notification plugin (#136) Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updating security filtering logic (#137) Signed-off-by: Ravi Thaluru <ravi1092@gmail.com> * Release note for 1.1.0.0 release. (#139) * Release note for 1.1.0.0 release. Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Correct copyright notices * Uses published daily snapshot dependencies (#141) Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * License header check (#142) * Provide default copyright header using IDE feature Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address #103 history write index is rolled over even if the history indices are disabled * Removed integtest.sh. (#148) Signed-off-by: dblock <dblock@dblock.org> * Adds mavenLocal back to repositories (#158) Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Making snapshot name to scripted input in template (#77) Signed-off-by: Ravi Thaluru <ravi1092@gmail.com> * Fix issues with security changes in rollup runnner (#161) * Updates index management version to 1.2 (#157) * Updates index management version to 1.2 * Updates job scheduler snapshot to 1.2 in test resources Signed-off-by: Robert Downs <downsrob@amazon.com> * Adds setting to search all rollup jobs on a target index (#165) * Adds cluster setting to search all rollup jobs Signed-off-by: Clay Downs <downsrob@amazon.com> * Adds implementation for the delay feature in rollup jobs (#147) * Adds delay implementation for rollup jobs * Removes non-continuous delay implementation * Adds additional rollup delay tests Signed-off-by: Clay Downs <downsrob@amazon.com> * Updates testCompile mockito version, adds AwaitsFix annotation to MetadataRegressionIT tests (#168) * Updates testCompile mockito version to match OpenSearch changes * AwaitsFix on the failing MetadataRegressionIT tests Signed-off-by: Robert Downs <downsrob@amazon.com> * Adds cluster setting to configure index state management jitter (#153) * Adds jitter cluster setting, sets jitter to 0 for ISM tests Signed-off-by: Clay Downs <downsrob@amazon.com> * Allows out of band rollovers on an index without causing ISM to fail (#180) * Allows out of band rollovers on an index without causing ISM to fail Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes detekt issue Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Remove policy API on read only indices (#182) * In explain API not showing the total count to all users (#185) Co-authored-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Aditya Jindal <13850971+aditjind@users.noreply.github.com> Co-authored-by: Bowen Lan <62091230+bowenlan-amzn@users.noreply.github.com> Co-authored-by: Ravi <6005951+thalurur@users.noreply.github.com> Co-authored-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> Co-authored-by: Daniel Doubrovkine (dB.) <dblock@dblock.org>
wuychn
pushed a commit
to ochprince/index-management
that referenced
this pull request
Mar 16, 2023
…pensearch-project#180) * Allows out of band rollovers on an index without causing ISM to fail Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes detekt issue Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Drew Baugher 46505179+dbbaughe@users.noreply.github.com
Issue #, if available:
#179
Description of changes:
Previously when a user would rollover an index that ISM is managing and waiting to rollover, ISM would enter a bad state and cause ISM to do multiple rollovers of aliases.
There was a fix put in place to block this from happening where we'd also check to confirm the job performing the rollover for a specific index also confirms that index is still the write alias and fail otherwise.
This is good to prevent the first issue, but now users have to deal with failed indices and use the skip rollover setting.
This allows out of band rollovers to happen without affecting ISM, ISM will check to see if the specific index has already been rolled over for the given alias, and if so treat it as successful.
CheckList:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.