Support rolling upgrades to OpenSearch 1.0 from ODFE 1.13 and ES 7.10.x

[dblock]

Upgrading your cluster from ElasticSearch to OpenSearch and OpenSearch Dashboards is just like upgrading your cluster between versions of ElasticSearch and Kibana. Specifically, OpenSearch supports rolling upgrades and restart upgrades from ElasticSearch 6.8.0 through ElasticSearch 7.10.2 and to OpenSearch 1.0. OpenSearch Dashboards supports restart upgrades from Kibana 6.8.0 through Kibana 7.10.2 and to OpenSearch Dashboards 1.0.

• OpenSearch Dashboards, see Support restart upgrades to OpenSearch Dashboards 1.0 from Kibana v7.10.x OpenSearch-Dashboards#334.
• OpenSearch Plugins, see Support rolling upgrades for OpenSearch plugins, and restart upgrades for OpenSearch Dashboards Plugins 1.0 opensearch-plugins#12.
• OpenSearch itself, see [BUG] Meta issue to track all places we have broken backwards compatibility in OpenSearch 1.0.0-beta1 #640.
• Upgrade/migration guide, Document/create/add a migration guide from ES 7.10.2 #801

We have asked the community regarding the upgrade path to OpenSearch. Please continue adding your comments.

Additional context

[saratvemulapalli]

For OpenSearch we still have to understand what we have broken and what needs to be backwards compatible.
For example plugins we have already figure out these are the ones to take care of: opensearch-project/opensearch-plugins#12

Here is an issue we opened to track them and I am confident this just scratching the surface: #640

[saratvemulapalli]

Pasting it back from the earlier issue since this is important.
These are all the items which we should consider while working on the drop in upgrades.

Ordering of upgrading
Naming conventions
Upgrade time-lag (aka downtime)
Replication
Security
Downstream Dependencies
Upgrade alerting and notifications
Limits
Performance
Availability
Latency
Compatibility Matrix
Downgrade
APIs

[nknize]

Can we convert this into a meta issue and track incremental issues the community can help pick off?

[dblock]

Can we convert this into a meta issue and track incremental issues the community can help pick off?

What's a "meta" issue? How do I convert this?

[nknize]

adding the list to the description like you did and then just label it as meta

[nknize]

Blocked by #801 awaiting feedback.

[dblock]

#801 is resolved, however we don't have documentation for migrating from Kibana OSS to OpenSearch Dashboards, for which I opened opensearch-project/documentation-website#71

[dblock]

Closing this with opensearch-project/documentation-website#71 resolved.

[shawnz]

Hi there, please also consider supporting upgrades directly from ES 7.10.x to OS 1.1. If this path is not supported it will be somewhat inconvenient for us as OS 1.0 is not available as an RPM distribution.

See also opensearch-project/OpenSearch-Dashboards#334 (comment)

[dblock]

Hi there, please also consider supporting upgrades directly from ES 7.10.x to OS 1.1. If this path is not supported it will be somewhat inconvenient for us as OS 1.0 is not available as an RPM distribution.

See also opensearch-project/OpenSearch-Dashboards#334 (comment)

AFAIK that's the plan, cc: @saratvemulapalli.

[shawnz]

Thank you for the notice. The reason why I was concerned is because I noticed this 1.0 version check in Dashboards: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/a60c4a3a2531442e085b80d25109d4cb563ea754/src/core/server/ui_settings/create_or_upgrade_saved_config/is_config_version_upgradeable.ts#L78

But I'm not sure if that is intended to be kept like this or not, or if it even affects this scenario.

[dblock]

@shawnz Kibana and hence Dashboards never supported rolling upgrades, only restart upgrades :( I had opened opensearch-project/OpenSearch-Dashboards#365 on the topic.

[dblock]

@shawnz Will you please open an issue in OpenSearch Dashboards pointing to that line, and specifically asking to "Support direct upgrades from Kibana 7.10.2"?

[shawnz]

Will do, thank you. My apologies if I am getting the terminology wrong but I don't mind having to do a whole cluster restart for now as long as the upgrade path is supported in some way or another.

[dblock]

Will do, thank you. My apologies if I am getting the terminology wrong but I don't mind having to do a whole cluster restart for now as long as the upgrade path is supported in some way or another.

No apology needed. I get confused all the time, too! Thank you.

[saratvemulapalli]

Hi there, please also consider supporting upgrades directly from ES 7.10.x to OS 1.1. If this path is not supported it will be somewhat inconvenient for us as OS 1.0 is not available as an RPM distribution.
See also opensearch-project/OpenSearch-Dashboards#334 (comment)

AFAIK that's the plan, cc: @saratvemulapalli.

Yup it is. OpenSearch will support rolling upgrades from ES 7.10.x to OpenSearch 1.1.0 directly via Snapshot, Restart and Rolling Upgrades.
As @dblock said, Dashboards have limitations with rolling upgrades.