[HLRC] Ensure that response parsers support unknown fields

[martijnvg]

In the HLRC code response parsers using object parsers need to support handeling of unknown field. Otherwise a released HLRC may fail parsing responses from ES nodes released after it. All HLRC areas need to be checked.

After a quick check the follow hlrc areas use ConstructingObjectParser with ignoreUnknownFields flag set to false:

• indices.getTemplate Update IndexTemplateMetaData to allow unknown fields #38448
• ingest.getPipeline Support unknown fields in ingest pipeline map configuration #38352
• snapshot.verifyRepository Update verify repository to allow unknown fields #37619
• rollup.getRollupCapabilities Update Rollup Caps to allow unknown fields #38339
• rollup.getRollupIndexCapabilities Update Rollup Caps to allow unknown fields #38339
• watcher.deleteWatch Update Delete Watch to allow unknown fields #37435
• watcher.putWatch Update Put Watch to allow unknown fields #37494
• watcher.executeWatch Update Execute Watch to allow unknown fields #37498
• ilm.getLifecyclePolicy Fix ILM Lifecycle Policy to allow unknown fields #38041
• ilm.lifecycleManagementStatus Fix ILM status to allow unknown fields #38043
• ilm.explainLifecycle Fix ILM explain response to allow unknown fields #38054
• security.getUsers Update get users to allow unknown fields #37593
• security.enableUser (EmptyResponse) Deprecate HLRC EmptyResponse used by security #37540
• security.disableUser (EmptyResponse) Deprecate HLRC EmptyResponse used by security #37540
• security.authenticate Update authenticate to allow unknown fields #37713
• security.getUserPrivileges wont fix re [HLRC] Ensure that response parsers support unknown fields #36938 (comment)
• security.clearRealmCache wont fix re [HLRC] Ensure that response parsers support unknown fields #36938 (comment)
• security.clearRolesCache wont fix re [HLRC] Ensure that response parsers support unknown fields #36938 (comment)
• security.changePassword (EmptyResponse) Deprecate HLRC EmptyResponse used by security #37540
• security.getRoles wont fix re [HLRC] Ensure that response parsers support unknown fields #36938 (comment)

The following use old school parsing and will (likely) fail with unknown fields. FWIW, some of these are just a guess as to how bad they will behave.

• bulk
• getMapping (has some asserts, could fail)
• snapshot.getRepository
• license.get (just returns whatever json it gets, so its questionable)
• security.putPrivileges

Disclaimer: It is possible that other old school parsers will quite easily bomb out given unknown fields. This list is likely not exhaustive depending on where the json fields are added.

Maybe we need make an automated check for this? (maybe forbidden apis that checks response classes in hlrc module only)

[elasticmachine]

Pinging @elastic/es-core-features

[hub-cap]

Ive gone thru all of our APIs and updated the list above with the full list of parsers (or nested parsers) that sets ignoreUnknownFields to false. These all need to be addressed. The second list is the old parsers that are not ObjectParsers that dont allow (or did not appear to allow unknown fields).

Please note I have added a bit of Disclaimer to the issue as well. I think that it would be easy to fail some of the parsers depending on where the extra fields are inserted. This would require a user to upgrade. I think as a fallback we should definitely record every time someone messes with the actions to add fields and signify they are breaking to the older versions of the HLRC.

I dont know how we could check that we are using the ConstructingObjectParser's with false at runtime... The only way we could do it is saying "you must use this constructor", but the 2 arg constructor already sets ignoreUnknownFields to false. We can at least put that one in forbiddenAPIs.