Introduce Environment variable for choosing serialisation - JDK or custom

[Dhruvan1217]

Is your feature request related to a problem?
Currently, In some cases, users are having problems (#4085) while upgrading from 2.10 to 2.11+ where it switches the serialisation to custom serialisation technique (Ref: #2780) and have shard allocation issues.

What solution would you like?

• We can introduce an environment variable like USE_JDK_SERIALIZATION=true (Boolean) to allow the users to choose what serialisation they want to choose until all the corner cases are sorted for using custom serialisation.
• Maybe default value for USE_JDK_SERIALIZATION is false if we think the custom serialisation is faster.

[cwperks]

Thank you for filing this request @Dhruvan1217. @parasjain1 do you have any concerns with introducing a setting (something like plugins.security.use_jdk_serialization) that will explicitly tell a cluster to use JDK serialization? There is a ConfigConstant for this, but it is not read in as a setting and used where the serialization format is determined.

[stephen-crawford]

[Triage] Hi @Dhruvan1217 thanks for filing this issue. @parasjain1 could you please provide your input on this request and make a contribution towards resolving the backwards compatibility issue. Otherwise we will have to pull the feature in the next version release.

[peternied]

I'll be looking into this, I don't think we should add a feature flag, but I'll address the upgrade problem