Migrated commons-lang usages to commons-lang3

[shigarg1]

Issue Link - #17130

[kgyrtkirk]

looks great! I hope the tests will pass :)
thank you for taking care of this :)

[clintropolis]

drive by comment: i'm pretty sure it is important that these are marked as scope provided so the jars don't get duplicated by the extensions when building a distribution. That said, i could also be misremembering, please make sure that there are not multiple copies after this change

[shigarg1]

I build the package with -Pdist. I checked for commons-lang jar in the distribution
I can only find 2 instances

➜ apache-druid-32.0.0-SNAPSHOT git:(commons-lang3) find . -name "commons-lang-*.jar"
./extensions/druid-ranger-security/commons-lang-2.6.jar
./lib/commons-lang-2.4.jar

[kgyrtkirk]

there are no places where only the scope gets removed:

• all pom.xml changes are either remove the dep
• there is one place where the dependency is upgraded from lang2 to lang3; but there are no scope changes there either

[kgyrtkirk]

there is check failing with some license issue

it seems like extensions-core/druid-ranger-security still refereces the old lib:

[INFO] org.apache.druid.extensions:druid-ranger-security:jar:32.0.0-SNAPSHOT
[INFO] +- org.apache.ranger:ranger-plugins-common:jar:2.4.0:compile
[INFO] |  +- commons-lang:commons-lang:jar:2.6:compile

[kgyrtkirk]

there are no places where only the scope gets removed:

• all pom.xml changes are either remove the dep
• there is one place where the dependency is upgraded from lang2 to lang3; but there are no scope changes there either

[shigarg1]

there is check failing with some license issue

it seems like extensions-core/druid-ranger-security still refereces the old lib:

[INFO] org.apache.druid.extensions:druid-ranger-security:jar:32.0.0-SNAPSHOT
[INFO] +- org.apache.ranger:ranger-plugins-common:jar:2.4.0:compile
[INFO] |  +- commons-lang:commons-lang:jar:2.6:compile

Let me add back the license for commons-lang? It can be removed once the dependencies also migrate to commons-lang3.

[kgyrtkirk]

there is no need to force-push the branch - the PR will be squashed in the end anyway.
taking a look at the last commit is something which is usefull;
force pushes are interfere with view reviewed changes ; and the "reviewed" checkmarks on the files tab....since I had your branch checked out earlier I was able to quickly diff the changes - but it would have been easier to instantly see that only the licenses.yaml was changed

[kgyrtkirk]

@shigarg1 it seems like you don't have any email addresses associated with you github account; so the system would use the noreply email address for the commit: 89958755+shigarg1@users.noreply.github.com

Let me know if that's ok - or you could add one by following this guide

[shigarg1]

@kgyrtkirk Updated the email.

[kgyrtkirk]

After some discussion with @clintropolis we've found that by fixing Druid to not use commons-lang2 ; in fact leads to older versions of commons-lang-2 to appear:

before/after commons-lang jar presences

dev@review:~/dist$ diff -U0  <(find orig/|cut -d/ -f2-|sort) <(find lang3/|cut -d / -f2-|sort)
--- /dev/fd/63	2024-09-27 08:41:47.230818152 +0000
+++ /dev/fd/62	2024-09-27 08:41:47.230818152 +0000
@@ -281 +281 @@
-apache-druid-32.0.0-SNAPSHOT/extensions/ambari-metrics-emitter/commons-lang-2.6.jar
+apache-druid-32.0.0-SNAPSHOT/extensions/ambari-metrics-emitter/commons-lang-2.5.jar
@@ -1018 +1017,0 @@
-apache-druid-32.0.0-SNAPSHOT/extensions/druid-redis-cache/commons-lang-2.6.jar
@@ -1194 +1193 @@
-apache-druid-32.0.0-SNAPSHOT/extensions/druid-thrift-extensions/commons-lang-2.6.jar
+apache-druid-32.0.0-SNAPSHOT/extensions/druid-thrift-extensions/commons-lang-2.4.jar
@@ -1383 +1382 @@
-apache-druid-32.0.0-SNAPSHOT/lib/commons-lang-2.6.jar
+apache-druid-32.0.0-SNAPSHOT/lib/commons-lang-2.4.jar

commons-lang:2.4 is being pulled into the sql module like this

parts of: mvn dependency:tree -pl sql

[INFO] +- org.apache.calcite:calcite-core:jar:1.37.0:compile
[INFO] |  +- net.hydromatic:aggdesigner-algorithm:jar:6.0:runtime
[INFO] |  |  \- commons-lang:commons-lang:jar:2.4:runtime

the net.hydromatic:aggdesigner-algorithm:jar:6.0:runtime is a pretty old artifact; and its only being used in the TileSuggester ; which is connected to Lattice ; I don't think we use those parts right now - so it could be excluded (along with lang-2)

retaining the lang-2.6 reference in the root pom.xml's dependencyManagement could ensure that other usages of lang-2 are upgraded to 2.6 (like before)

new diff

dev@review:~/dist$ diff -U0  <(find orig/|cut -d/ -f2-|sort) <(find lang3/|cut -d / -f2-|sort)
--- /dev/fd/63	2024-09-27 09:04:00.812205455 +0000
+++ /dev/fd/62	2024-09-27 09:04:00.812205455 +0000
@@ -1018 +1017,0 @@
-apache-druid-32.0.0-SNAPSHOT/extensions/druid-redis-cache/commons-lang-2.6.jar
@@ -1349 +1347,0 @@
-apache-druid-32.0.0-SNAPSHOT/lib/aggdesigner-algorithm-6.0.jar
@@ -1383 +1380,0 @@
-apache-druid-32.0.0-SNAPSHOT/lib/commons-lang-2.6.jar

proposed changes diff

diff --git pom.xml pom.xml
index 96dbfa6004..4fd3ca2a28 100644
--- pom.xml
+++ pom.xml
@@ -301,6 +301,11 @@
                 <artifactId>commons-logging</artifactId>
                 <version>1.1.1</version>
             </dependency>
+            <dependency>
+                <groupId>commons-lang</groupId>
+                <artifactId>commons-lang</artifactId>
+                <version>2.6</version>
+            </dependency>
            <dependency>
                <groupId>commons-net</groupId>
                <artifactId>commons-net</artifactId>
@@ -503,6 +508,10 @@
                     <groupId>com.google.guava</groupId>
                     <artifactId>guava</artifactId>
                   </exclusion>
+                  <exclusion>
+                    <groupId>net.hydromatic</groupId>
+                    <artifactId>aggdesigner-algorithm</artifactId>
+                  </exclusion>
                 </exclusions>
             </dependency>
             <dependency>

taking a look at all the jars in the dist tgz (-Pdist -Pbundle-contrib-exts); there are quite a few more deps for which different versions are present: protobuf,okhttp,okio,orc-core,kafka-clients ... and many more 😱

find lang3/|grep jar|sed -r 's|.*/||'|sort|uniq

[kgyrtkirk]

Thank you @shigarg1!