Allow follower indices to override leader settings #58103
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today when creating a follower index via the put follow API, or via an auto-follow pattern, it is not possible to specify settings overrides for the follower index. Instead, we copy all of the leader index settings to the follower. Yet, there are cases where a user would want some different settings on the follower index such as the number of replicas, or allocation settings. This commit addresses this by allowing the user to specify settings overrides when creating follower index via manual put follower calls, or via auto-follow patterns. Note that not all settings can be overrode (e.g., index.number_of_shards) so we also have detection that prevents attempting to override settings that must be equal between the leader and follow index. Note that we do not even allow specifying such settings in the overrides, even if they are specified to be equal between the leader and the follower index. Instead, the must be implicitly copied from the leader index, not explicitly set by the user.
jasontedor
added
>enhancement
:Distributed Indexing/CCR
|
Pinging @elastic/es-distributed (:Distributed/CCR) |
elasticmachine
added
the
Team:Distributed (Obsolete)
jasontedor
commented
Jun 15, 2020
| .put(IndexMetadata.SETTING_INDEX_PROVIDED_NAME, request.getFollowerIndex()) | ||
| .put(CcrSettings.CCR_FOLLOWING_INDEX_SETTING.getKey(), true); | ||
| .put(CcrSettings.CCR_FOLLOWING_INDEX_SETTING.getKey(), true) | ||
| .put(request.getSettings()) |
There was a problem hiding this comment.
This is the crux of the change, where the request settings are propagated to the follower index. The rest of this change is just ceremony around this, parsing the settings from the request, validating that they are not including any settings that must be propagated from the leader index, and tests around this.
|
@elasticmachine update branch |
dnhatn
approved these changes
Jun 18, 2020
jasontedor
added a commit
that referenced
this pull request
Jun 18, 2020
Today when creating a follower index via the put follow API, or via an auto-follow pattern, it is not possible to specify settings overrides for the follower index. Instead, we copy all of the leader index settings to the follower. Yet, there are cases where a user would want some different settings on the follower index such as the number of replicas, or allocation settings. This commit addresses this by allowing the user to specify settings overrides when creating follower index via manual put follower calls, or via auto-follow patterns. Note that not all settings can be overrode (e.g., index.number_of_shards) so we also have detection that prevents attempting to override settings that must be equal between the leader and follow index. Note that we do not even allow specifying such settings in the overrides, even if they are specified to be equal between the leader and the follower index. Instead, the must be implicitly copied from the leader index, not explicitly set by the user.
29 tasks
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Aug 6, 2020
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Aug 18, 2020
github-actions bot
pushed a commit
to elastic/elasticsearch-net
that referenced
this pull request
Aug 18, 2020
github-actions bot
pushed a commit
to elastic/elasticsearch-net
that referenced
this pull request
Aug 18, 2020
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Aug 18, 2020
…Is (#4947) (#4966) Relates: elastic/elasticsearch#58103 Co-authored-by: Russ Cam <russ.cam@elastic.co>
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Aug 18, 2020
…Is (#4947) (#4965) Relates: elastic/elasticsearch#58103 Co-authored-by: Russ Cam <russ.cam@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today when creating a follower index via the put follow API, or via an auto-follow pattern, it is not possible to specify settings overrides for the follower index. Instead, we copy all of the leader index settings to the follower. Yet, there are cases where a user would want some different settings on the follower index such as the number of replicas, or allocation settings. This commit addresses this by allowing the user to specify settings overrides when creating follower index via manual put follower calls, or via auto-follow patterns. Note that not all settings can be overrode (e.g., index.number_of_shards) so we also have detection that prevents attempting to override settings that must be equal between the leader and follow index. Note that we do not even allow specifying such settings in the overrides, even if they are specified to be equal between the leader and the follower index. Instead, the must be implicitly copied from the leader index, not explicitly set by the user.