Multiple LICENSE files present in some jars

[zabetak]

Search before asking

• I had searched in the issues and found no similar issues.

Please describe the bug 🐞

Some jars from those listed contain multiple LICENSE files, which are sometimes redundant and also create confusion.

xtable-api-0.3.0-incubating.jar
xtable-aws-0.3.0-incubating-bundled.jar
xtable-aws-0.3.0-incubating.jar
xtable-core_2.12-0.3.0-incubating.jar
xtable-hive-metastore-0.3.0-incubating-bundled.jar
xtable-hive-metastore-0.3.0-incubating.jar
xtable-hudi-support-extensions_2.12-0.3.0-incubating-bundled.jar
xtable-hudi-support-extensions_2.12-0.3.0-incubating.jar
xtable-hudi-support-utils-0.3.0-incubating.jar 

I checked using the following command:

for i in `ls *jar`;do  echo $i; jar tf $i | grep LICENSE; done

One common problem in most jars is that they have the following structure:

META-INF/LICENSE
META-INF/LICENSE-bundled
META-INF/licenses/

The LICENSE-bundled and licenses directory are clearly redundant for non-bundled jars and the opposite holds for bundled jars. In all cases there should be only one LICENSE file directly under META-INF directory.

The most problematic of all is xtable-hive-metastore-0.3.0-incubating-bundled.jar which contains lots of licenses in non-canonical places making it very hard to review. As before there should be only one main license file (i.e., META-INF/LICENSE) that lists all 3rd party code and necessary licenses should be only under (i.e., META-INF/licenses/ directory).

Are you willing to submit PR?

• I am willing to submit a PR!
• I am willing to submit a PR but need help getting started!

Code of Conduct

• I agree to follow this project's Code of Conduct

[vinishjail97]

Based on vote thread from @jbonofre.

- NOK: DISCLAIMER-WIP is here, but it would be great to list why it's
still WIP (for instance, "Contains code that may not be compatible
with the Apache License" or so). Else it should be DISCLAIMER.
- NOK: LICENSE is there in the source distribution, but it should
mention that xtable-core/src/main/java/org/apache/xtable/hudi/HudiInstantUtils.java
contains code directly copied from
org.apache.hudi.common.table.timeline.HoodieActiveTimeline

hudi is a top-level ASF project and this guidance from ASF team mentions it's not recommended to include these dependencies. @zabetak @jbonofre Is this okay?
https://issues.apache.org/jira/browse/LEGAL-684

- NOK: xtable-aws bundle jar includes a LICENSE file but AWS SDK is
not documented here. NOTICE should also include AWS SDK NOTICE (as
it's ALv2 dep).
- NOK: xtable-hive-metastore jar includes a LICENSE file but Guava,
Avro, Scala, Caffeine, RoaringBitmap, HttpClient, RocksDB, HBase,
javassist are not listed (same for the NOTICE file).
- NOK: xtable-hudi-support extension jar includes a LICENSE but
Jackson, log4j, guava, scala, j2objc, iceberg, delta, spark-sql,
caffeine, RoaringBitmap are not listed (same for the NOTICE file).

I will put up a PR for these license updates.

[zabetak]

Regarding HudiInstantUtils.java I prefer to not include it as explained in LEGAL-684.

Regarding the missing licenses, I raised #700.

This issue is more about the redundant LICENSE files that are present in some jars notably the hive-metastore one.

[vinishjail97]

@zabetak @jbonofre I have pushed the PR to update license file contents. Based on the legal guidance, we don't want to bloat the NOTICE file with these contents is what I have understood. I have removed the references for dependent license files as it was not followed in other ASF projects.

#707

[zabetak]

Just to clarify a bit more the problem described here, I will use some concrete examples.

xtable-aws-0.3.0-incubating.jar

Actual structure

META-INF/licenses/
META-INF/LICENSE
META-INF/LICENSE-bundled
META-INF/licenses/LICENSE-antlr4-runtime
META-INF/licenses/LICENSE-checker-qual
META-INF/licenses/LICENSE-glassfish-java-servlet-jsp
META-INF/licenses/LICENSE-javax-annotation-api
META-INF/licenses/LICENSE-paranamer
META-INF/licenses/LICENSE-slf4j-api

Expected structure

META-INF/LICENSE

xtable-aws-0.3.0-incubating-bundled.jar

Actual structure

META-INF/LICENSE-bundled
META-INF/licenses/
META-INF/licenses/LICENSE-antlr4-runtime
META-INF/licenses/LICENSE-checker-qual
META-INF/licenses/LICENSE-glassfish-java-servlet-jsp
META-INF/licenses/LICENSE-javax-annotation-api
META-INF/licenses/LICENSE-paranamer
META-INF/licenses/LICENSE-slf4j-api
META-INF/LICENSE

Expected structure

META-INF/licenses/
META-INF/licenses/LICENSE-antlr4-runtime
META-INF/licenses/LICENSE-checker-qual
META-INF/licenses/LICENSE-glassfish-java-servlet-jsp
META-INF/licenses/LICENSE-javax-annotation-api
META-INF/licenses/LICENSE-paranamer
META-INF/licenses/LICENSE-slf4j-api
META-INF/licenses/LICENSE-thoughtworks
META-INF/licenses/LICENSE-protobuf
META-INF/licenses/LICENSE-jcodings
META-INF/licenses/LICENSE-joni
META-INF/licenses/LICENSE-jaxb-impl
META-INF/licenses/LICENSE-jamon
META-INF/licenses/LICENSE-javassist
META-INF/LICENSE <-- The content is that of LICENSE-bundled

Much more work is needed for xtable-hive-metastore-0.3.0-incubating-bundled.jar and maybe others.

[piyushdubey]

@zabetak I opened the above PR to address the issue where LICENSE-bundled and NOTICE-bundled files were being copied into the artifact. The fix ensures that only the canonical LICENSE and NOTICE files are included. Could you please review?