BACKPORT Security Tokens moved to a new separate index #41673
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces the `.security-tokens` and `.security-tokens-7` alias-index pair. Because index snapshotting is at the index level granularity (ie you cannot snapshot a subset of an index) snapshoting .`security` had the undesirable effect of storing ephemeral security tokens. The changes herein address this issue by moving tokens "seamlessly" (without user intervention) to another index, so that a "Security Backup" (ie snapshot of `.security`) would not be bloated by ephemeral data.
albertzaharovits
added
:Security/Authentication
|
Pinging @elastic/es-security |
albertzaharovits
changed the title
This was referenced Apr 30, 2019
albertzaharovits
added a commit
that referenced
this pull request
Apr 30, 2019
albertzaharovits
added a commit
that referenced
this pull request
May 1, 2019
This fixes a low level bug that manifests, in certain circumstances, by the failure of the refresh operation. Version 7.1 added a new `superseded_by` field to the `.security` index mapping. This field is used when indexing a refresh operation (a document update). Because the document update was not guarded by the obligatory `prepareIndexIfNeededThenExecute` the refresh operation would fail if it were the first operation when the cluster was upgraded from a version < 7.1 . This failure was catched (and fails reliably) in the backport #41673 .
) This fixes a low level bug that manifests, in certain circumstances, by the failure of the refresh operation. Version 7.1 added a new `superseded_by` field to the `.security` index mapping. This field is used when indexing a refresh operation (a document update). Because the document update was not guarded by the obligatory `prepareIndexIfNeededThenExecute` the refresh operation would fail if it were the first operation when the cluster was upgraded from a version < 7.1 . This failure was catched (and fails reliably) in the backport elastic#41673 .
|
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request-1/13111/ elasticsearch-ci/1 blip failure ascribed to #41549 |
albertzaharovits
added a commit
that referenced
this pull request
May 2, 2019
akhil10x5
pushed a commit
to akhil10x5/elasticsearch
that referenced
this pull request
May 2, 2019
Mutes security tokens BWC tests for the elastic#41673 backport.
akhil10x5
pushed a commit
to akhil10x5/elasticsearch
that referenced
this pull request
May 2, 2019
) This fixes a low level bug that manifests, in certain circumstances, by the failure of the refresh operation. Version 7.1 added a new `superseded_by` field to the `.security` index mapping. This field is used when indexing a refresh operation (a document update). Because the document update was not guarded by the obligatory `prepareIndexIfNeededThenExecute` the refresh operation would fail if it were the first operation when the cluster was upgraded from a version < 7.1 . This failure was catched (and fails reliably) in the backport elastic#41673 .
akhil10x5
pushed a commit
to akhil10x5/elasticsearch
that referenced
this pull request
May 2, 2019
Reverts the mutes from elastic#41698 .
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this pull request
May 27, 2019
Mutes security tokens BWC tests for the elastic#41673 backport.
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this pull request
May 27, 2019
) This fixes a low level bug that manifests, in certain circumstances, by the failure of the refresh operation. Version 7.1 added a new `superseded_by` field to the `.security` index mapping. This field is used when indexing a refresh operation (a document update). Because the document update was not guarded by the obligatory `prepareIndexIfNeededThenExecute` the refresh operation would fail if it were the first operation when the cluster was upgraded from a version < 7.1 . This failure was catched (and fails reliably) in the backport elastic#41673 .
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this pull request
May 27, 2019
Reverts the mutes from elastic#41698 .
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #40742
This commit introduces the .security-tokens and .security-tokens-7 alias-index pair. Because index snapshotting is at the index level granularity (ie you cannot snapshot a subset of an index) snapshoting .security had the undesirable effect of storing ephemeral security tokens. The changes herein address this issue by moving tokens "seamlessly" (without user intervention) to another index, so that a "Security Backup" (ie snapshot of .security) would not be bloated by ephemeral data.
There is no change in field mappings, but the tokens mappings have been copy-pasted in a dedicated mapping file.
The rolling upgrade situation is trappy. This is because newly created tokens (creation or refresh) should be usable by both versions in the rolling upgrade situation. A big part of the change is due to this. For example, token docs will be generated in the .security index, until all nodes have been upgraded. Another situation is when we have to refresh a token that was generated by the previous version; in this case the "superseding" doc will be in the new index. I have yet to include this cases in the tests, but I plan to! I will drop comments about newly added tests.
Obsoletes #37236
Relates #34454