[SPARK-29557][BUILD] Update dropwizard/codahale metrics library to 3.2.6

[LucaCanali]

What changes were proposed in this pull request?

This proposes to update the dropwizard/codahale metrics library version used by Spark to 3.2.6 which is the last version supporting Ganglia.

Why are the changes needed?

Spark is currently using Dropwizard metrics version 3.1.5, a version that is no more actively developed nor maintained, according to the project's Github repo README.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing tests + manual tests on a YARN cluster.

[dongjoon-hyun]

ok to test

[SparkQA]

Test build #112480 has finished for PR 26212 at commit a77a273.

• This patch fails build dependency tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #112482 has finished for PR 26212 at commit a751820.

• This patch fails build dependency tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Hi, @LucaCanali . You need to run the following to update the dependency manifest.

$ dev/test-dependencies.sh --replace-manifest

[SparkQA]

Test build #112514 has finished for PR 26212 at commit 77c92a7.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Could you take a look at the build error? Apache Spark needs to support Ganglia, but metrics-ganglia:4.1.1 doesn't exist.

sbt.ResolveException: unresolved dependency: io.dropwizard.metrics#metrics-ganglia;4.1.1: not found

It seems 3.2.6 was the last version. Do we need to add another dependency?

• https://mvnrepository.com/artifact/io.dropwizard.metrics/metrics-ganglia

[dongjoon-hyun]

Ur, it seems due to this. They officially dropped this at 4.0.

• metrics-ganglia 4.0.2 not available in Maven repos dropwizard/metrics#1319

[LucaCanali]

Indeed, the latest version of metrics-ganglia is 3.2.6. I guess we could update metrics-ganglia to that version and just have it "stuck there", that is using a different version than the other metrics-* components?

[SparkQA]

Test build #112515 has finished for PR 26212 at commit 803a174.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[LucaCanali]

Another thought is that if the GangliaReporter is needed for Apache Spark in the coming years and the Dropwizard community dropped it in recent versions, maybe the Ganglia reporter can be supported by Apache Spark in external/spark-ganglia-lgpl?

[srowen]

I would just update to the last version with Ganglia support here. I don't know if we want to support it ourselves if there's not yet a compelling reason to use 4.x.

[LucaCanali]

OK for upgrading to a version with Ganglia support. I guess this discussion about upgrading to metrics versions 4.x or higher may come back at a later stage, I left a note in the JIRA too.
As for adding LICENCES files, I am not sure if it aplies here, as we are just bumping up versions of libraries already in use?

[srowen]

Yeah disregard that, this change no longer brings in a new component.

[srowen]

Looks OK pending tests

[SparkQA]

Test build #112549 has finished for PR 26212 at commit b22dba0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

+1, LGTM. Thank you, @LucaCanali and @srowen .