[BUG] Meta issue to track all places we have broken backwards compatibility in OpenSearch 1.0.0-beta1

[saratvemulapalli]

Describe the bug
The intent of this issue is to understand what we have broken and what problems should we be looking at in order solve a drop in replacement: #638

Feel free to add all of the breaking changes you see across the repo.

• opensearch.yml
• opensearch.cgroups.hierarchy.override
• opensearch.path.home
• opensearch.path.conf
• opensearch.distribution.type
• opensearch.networkaddress
• opensearch.transport.cname_in_publish_address

[dblock]

All these should be falling back to their previous names.

[mch2]

assign to me, will take this

[dblock]

We decided that OpenSearch will raise an error if both elasticsearch.yml and opensearch.yml are present to prevent users like myself from making mistakes.

For other changes, let's understand what they are, how those settings are used, and what a failure scenario looks like. @mch2

[mch2]

These three params are used to bootstrap the env and are set at startup from $OPENSEARCH env vars. If completely missing, the service won't start. If ES* equivalents also existed, which I'm not sure is a valid use case, their values would be ignored. At a minimum, we can detect & log a warning when both exist.

opensearch.path.home
opensearch.path.conf
opensearch.distribution.type - won't break if unset, defaults to 'unknown'.

These are my assumptions of the others listed from reading through the repo. Working on validating these assumptions.

Param	Description	Ref	Failure	Should warn?
opensearch.cgroups.hierarchy.override	undocumented JVM param - Only set when running in docker container inside opensearch-env to override cgroup path from /proc/self/cgroup to root.	read here set here	N/A, this is always set when running with docker	No
opensearch.networkaddress	.cache.ttl & .cache.negative.ttl - Sets dns cache tll	sets defaults	silent - would fallback to defaults	Yes
opensearch.transport.cname_in_publish_address	Used to format the publish_address field to include hostname [hostname/ip:port] instead of [Ip:port] default = false.	read	silent - host string would be missing	Yes

[dblock]

Since these are set from ENV, we should be talking about those OPENSEARCH_ ENV variables, right? Those are renames of ES_ ones?

[mch2]

Yep. I'm thinking at startup search for the presence of ES_ vars to issue a warning and abort for a subset of critical vars.

[mch2]

I'm not sure we are approaching this correctly - @nknize should we wait for concrete bwc bugs instead of issuing warnings/fallbacks?

[adnapibar]

IMO we should be providing a migration tool that interactively/quietly migrates to the OpenSearch 1.x from a supported ES version. This tool should be included in the bin directory of the OpenSearch distribution. I don't think throwing warnings or errors in the presence of an existing Elasticsearch version is any helpful to the user unless we can provide them a tool to fix it automatically for them.

[nknize]

IMO we should be providing a migration tool that interactively/quietly migrates to the OpenSearch 1.x from a supported ES version. This tool should be included in the bin directory of the OpenSearch distribution. I don't think throwing warnings or errors in the presence of an existing Elasticsearch version is any helpful to the user unless we can provide them a tool to fix it automatically for them.

+1 Let's keep them separate and not introduce more logic in the main server implementation that will only be carried in the 1.x release line.

[saratvemulapalli]

Agreed, I like what @adnapibar said.
This is something which is needed only for customers moving from Legacy versions of ES to OpenSearch 1.0.0 or 1.x.

For everything which is configuration based we should use the script to take care of it.
For everything which needs while rolling upgrades (i.e runtime) needs to be in the code. I would rather have it as something pluggable (maybe plugin? and not module) with the repo as it is only for 1.x line.

[dblock]

I have split this into 3 issues: #780, #781, #782 and #785, cc: @CEHENKLE

[dblock]

What do we want to do with this issue?