Skip to content

Strip index blocks from settings for reindex targets #80887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

williamrandolph
Copy link
Contributor

When migrating system features, we copy settings from old indices into
the new indices we create before reindexing. However, if we happen to
copy a write block, this causes the reindexing to fail.

Here, we strip the index block settings before applying settings to new
indices.

Fixes #80654

When migrating system features, we copy settings from old indices into
the new indices we create before reindexing. However, if we happen to
copy a write block, this causes the reindexing to fail.

Here, we strip the index block settings before applying settings to new
indices.
@williamrandolph williamrandolph added >bug :Core/Infra/Core Core issues without another label v8.0.0 auto-backport Automatically create backport pull requests when merged v7.16.1 labels Nov 19, 2021
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Nov 19, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

Copy link
Contributor

@gwbrown gwbrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the quick turnaround on this!

@gwbrown
Copy link
Contributor

gwbrown commented Nov 29, 2021

@elasticmachine update branch

@gwbrown gwbrown added auto-backport-and-merge auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) and removed auto-backport Automatically create backport pull requests when merged labels Nov 29, 2021
@gwbrown
Copy link
Contributor

gwbrown commented Nov 29, 2021

Failure is #66495.

@elasticmachine run elasticsearch-ci/part-2

@gwbrown gwbrown self-assigned this Nov 29, 2021
@gwbrown
Copy link
Contributor

gwbrown commented Nov 29, 2021

Something fell over while trying to set up an integration test cluster, looks like an intermittent failure?

@elasticmachine run elasticsearch-ci/part-2

@elasticsearchmachine elasticsearchmachine merged commit 43e6cac into elastic:master Nov 29, 2021
williamrandolph added a commit to williamrandolph/elasticsearch that referenced this pull request Nov 29, 2021
When migrating system features, we copy settings from old indices into
the new indices we create before reindexing. However, if we happen to
copy a write block, this causes the reindexing to fail. Here, we strip
the index block settings before applying settings to new indices. Fixes
elastic#80654
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.0
7.16 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 80887

gwbrown pushed a commit to gwbrown/elasticsearch that referenced this pull request Nov 29, 2021
When migrating system features, we copy settings from old indices into
the new indices we create before reindexing. However, if we happen to
copy a write block, this causes the reindexing to fail. Here, we strip
the index block settings before applying settings to new indices. Fixes
elasticsearchmachine pushed a commit that referenced this pull request Nov 29, 2021
When migrating system features, we copy settings from old indices into
the new indices we create before reindexing. However, if we happen to
copy a write block, this causes the reindexing to fail. Here, we strip
the index block settings before applying settings to new indices. Fixes
#80654
elasticsearchmachine pushed a commit that referenced this pull request Nov 29, 2021
When migrating system features, we copy settings from old indices into
the new indices we create before reindexing. However, if we happen to
copy a write block, this causes the reindexing to fail. Here, we strip
the index block settings before applying settings to new indices. Fixes

Co-authored-by: William Brafford <william.brafford@elastic.co>
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Nov 30, 2021
* upstream/master: (150 commits)
  Fix ComposableIndexTemplate equals when composed_of is null (elastic#80864)
  Optimize DLS bitset building for matchAll query (elastic#81030)
  URL option for BaseRunAsSuperuserCommand (elastic#81025)
  Less Verbose Serialization of Snapshot Failure in SLM Metadata (elastic#80942)
  Fix shadowed vars pt7 (elastic#80996)
  Fail shards early when we can detect a type missmatch (elastic#79869)
  Delegate Ref Counting to ByteBuf in Netty Transport (elastic#81096)
  Clarify `unassigned.reason` docs (elastic#81017)
  Strip blocks from settings for reindex targets (elastic#80887)
  Split off the values supplier for ScriptDocValues (elastic#80635)
  [ML] Switch message and detail for model snapshot deprecations (elastic#81108)
  [DOCS] Update xrefs for snapshot restore docs (elastic#81023)
  [ML] Updates visiblity of validate API (elastic#81061)
  Track histogram of transport handling times (elastic#80581)
  [ML] Fix datafeed preview with remote indices (elastic#81099)
  [ML] Fix acceptable model snapshot versions in ML deprecation checker (elastic#81060)
  [ML] Add logging for failing PyTorch test (elastic#81044)
  Extending the timeout waiting for snapshot to be ready (elastic#81018)
  [ML] Fix incorrect logging of unexpected model size error (elastic#81089)
  [ML] Make inference timeout test more reliable (elastic#81094)
  ...

# Conflicts:
#	server/src/main/java/org/elasticsearch/index/mapper/NumberFieldMapper.java
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Nov 30, 2021
* upstream/master: (55 commits)
  Fix ComposableIndexTemplate equals when composed_of is null (elastic#80864)
  Optimize DLS bitset building for matchAll query (elastic#81030)
  URL option for BaseRunAsSuperuserCommand (elastic#81025)
  Less Verbose Serialization of Snapshot Failure in SLM Metadata (elastic#80942)
  Fix shadowed vars pt7 (elastic#80996)
  Fail shards early when we can detect a type missmatch (elastic#79869)
  Delegate Ref Counting to ByteBuf in Netty Transport (elastic#81096)
  Clarify `unassigned.reason` docs (elastic#81017)
  Strip blocks from settings for reindex targets (elastic#80887)
  Split off the values supplier for ScriptDocValues (elastic#80635)
  [ML] Switch message and detail for model snapshot deprecations (elastic#81108)
  [DOCS] Update xrefs for snapshot restore docs (elastic#81023)
  [ML] Updates visiblity of validate API (elastic#81061)
  Track histogram of transport handling times (elastic#80581)
  [ML] Fix datafeed preview with remote indices (elastic#81099)
  [ML] Fix acceptable model snapshot versions in ML deprecation checker (elastic#81060)
  [ML] Add logging for failing PyTorch test (elastic#81044)
  Extending the timeout waiting for snapshot to be ready (elastic#81018)
  [ML] Fix incorrect logging of unexpected model size error (elastic#81089)
  [ML] Make inference timeout test more reliable (elastic#81094)
  ...
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Nov 30, 2021
* upstream/master: (55 commits)
  Fix ComposableIndexTemplate equals when composed_of is null (elastic#80864)
  Optimize DLS bitset building for matchAll query (elastic#81030)
  URL option for BaseRunAsSuperuserCommand (elastic#81025)
  Less Verbose Serialization of Snapshot Failure in SLM Metadata (elastic#80942)
  Fix shadowed vars pt7 (elastic#80996)
  Fail shards early when we can detect a type missmatch (elastic#79869)
  Delegate Ref Counting to ByteBuf in Netty Transport (elastic#81096)
  Clarify `unassigned.reason` docs (elastic#81017)
  Strip blocks from settings for reindex targets (elastic#80887)
  Split off the values supplier for ScriptDocValues (elastic#80635)
  [ML] Switch message and detail for model snapshot deprecations (elastic#81108)
  [DOCS] Update xrefs for snapshot restore docs (elastic#81023)
  [ML] Updates visiblity of validate API (elastic#81061)
  Track histogram of transport handling times (elastic#80581)
  [ML] Fix datafeed preview with remote indices (elastic#81099)
  [ML] Fix acceptable model snapshot versions in ML deprecation checker (elastic#81060)
  [ML] Add logging for failing PyTorch test (elastic#81044)
  Extending the timeout waiting for snapshot to be ready (elastic#81018)
  [ML] Fix incorrect logging of unexpected model size error (elastic#81089)
  [ML] Make inference timeout test more reliable (elastic#81094)
  ...
@danhermann danhermann added v7.16.0 and removed v7.16.1 labels Dec 3, 2021
@williamrandolph williamrandolph deleted the si/strip-blocks-for-reindex-targets branch May 23, 2022 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >bug :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team v7.16.0 v8.0.0 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrating Kibana system features from 6.8.20 fails with reindexing error
5 participants