Write a multinode qa test for ingest

[javanna]

Ingest has a proxy filter that reroutes ingest requests to ingest nodes when they are sent to nodes with node.ingest set to false. The filter is unit tested but we could definitely use some more testing. A qa test would be perfect but we would need to be able to set different node settings per node, which is something not yet possible in our qa test infra, which will be possible soon. Once we have that we should go ahead and write a qa multinode test for this.

[martijnvg]

+1 we should have a multimode qa test. We can then also test the scenario where one node has the an ingest plugin installed and the other hasn't. I'm pretty sure sure that we don't fail nicely here. We use geoip as plugin to test with.

[martijnvg]

I quickly looked at this and starting a multi node qa test is easy. In order to test the scenario one node needs to be started with ingest enabled (using default node settings) and one node with ingest disabled (node.ingest: false), but setting specific settings per node doesn't seem possible.

[javanna]

yea that is why I opened this issue, we have to wait for test infra improvements that are in the work to have different settings on each node. I discussed this with @rjernst .

[talevy]

Aren't we achieving this by leveraging the nodeOrdinal in the test?

On this note,
I have begun exploring this while adding tests to the new pipeline validation. I am attempting to set up a cluster where one node has a certain plugin installed, while the other does not. From the way I see it, nodePlugins method can take in a nodeOrdinal just like the settings.
what do you think?

[martijnvg]

Aren't we achieving this by leveraging the nodeOrdinal in the test?

Right not this is the only way how we can test it and I think for now this okay. However because we're extending from ESIntegTestCase we setup the nodes in a non realistic setup. (we use random settings that are unlikely to be used in production and the nodes aren't started from as one would normally do in production (using one of our assemblies and start it from the command line) ) So ideally this should be tested in a qa scenario, but can't yet customise the cluster to run with different plugins)

[clintongormley]

Given that ingest is now a module, is this test still required?

[martijnvg]

@clintongormley This issue is about adding a qa test that verifies that the redirecting works as expected when some nodes have ingest enabled and some don't. We have a Java integration test for this, but that isn't ideal as it doesn't test how users run ES. Also we have a qa test that test when all nodes have ingest disabled, but running some nodes with ingest enabled and some without isn't possible at the moment.

[talevy]

Coming back to this... do we have other examples of us wanting to do this?

I feel like the Bwc cluster formation code that has been since added to groovy can be modified to make solve this issue. (

elasticsearch/buildSrc/src/main/groovy/org/elasticsearch/gradle/test/ClusterFormationTasks.groovy

Line 483 in c15e5d4

static Task configureInstallPluginTask(String name, Project project, Task setup, NodeInfo node, Project plugin, String prefix) {

)

any objections to proceeding with this?

[rjernst]

I'm not sure the reasoning for this test is still relevant. I believe the ingest proxy was removed through various refactorings. What benefit is there then to having an ingest specific multi node test? I think having qa tests for various configurations of ES and running rest tests or whatever on them would be good as a high level check, but feature specific qa tests are just more noise. If ingest lacks some form of testing, let's add more unit tests.

[talevy]

@rjernst to be honest, I am OK that with that decision as well. Just hoping to get a decision out of this since we have been seemingly OK without this for two years.

[talevy]

No longer needed, closing