[SPARK-44811][BUILD] Upgrade Guava to 33.2.1-jre #42493

pan3793 · 2023-08-15T02:07:10Z

What changes were proposed in this pull request?

This PR upgrades Spark's built-in Guava from 14 to 33.2.1-jre

Currently, Spark uses Guava 14 because the previous built-in Hive 2.3.9 is incompatible with new Guava versions. HIVE-27560 (apache/hive#4542) makes Hive 2.3.10 compatible with Guava 14+ (thanks to @LuciferYang)

Why are the changes needed?

It's a long-standing issue, see prior discussions at #35584, #36231, and #33989

Does this PR introduce any user-facing change?

Yes, some user-faced error messages changed.

How was this patch tested?

GA passed.

pom.xml

LuciferYang · 2023-08-15T03:49:13Z

Thank you for your efforts on this. @pan3793

pan3793 · 2023-08-15T10:32:01Z

cc @sunchao @JoshRosen @HyukjinKwon @dongjoon-hyun @srowen @mridulm, what do you think about this approach?

srowen · 2023-08-15T12:18:50Z

We should probably do something like this for 4.0, yes.
Why does it require removing the isolated classloader? That's the only part I'm concerned about breaking something else.

pan3793 · 2023-08-15T12:34:48Z

We should probably do something like this for 4.0, yes.

@srowen yes, it's target to 4.0

Why does it require removing the isolated classloader?

Guava is marked as shared, which means that Guava is not isolated totally. (I don't fully understand the background, but the fact is, it's required, excluding it from shared classes breaks the test)

https://github.com/apache/spark/blob/afcccb42c96cc7785a85c9463523f34d7a900a1d/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala#L205C3-L219

So technically, if we want to upgrade Guava while keeping IsolatedClientLoader, we should make all Hive versions compatible with the new Guava, it's impossible.

And as I comment in https://www.mail-archive.com/dev@spark.apache.org/msg30708.html, I think the IsolatedClientLoader is not required after dropping Java 8

Dropping support for Java 8 means dropping support for Hive lower than
2.0(exclusive)[1].

IsolatedClientLoader aims to allow using different Hive jars to communicate
with different versions of HMS. AFAIK, the current built-in Hive 2.3.9 client
works well on communicating with the Hive Metastore server through 2.1 to 3.1
(maybe 2.0 too, not sure). This brings a new question, is the
IsolatedClientLoader required then?

I think we should drop IsolatedClientLoader because

As explained above, we can use the built-in Hive 2.3.9 client to communicate
with HMS 2.1+

Since SPARK-42539[2], the default Hive 2.3.9 client does not use
IsolatedClientLoader, and as explained in SPARK-42539, IsolatedClientLoader
causes some inconsistent behaviors.

It blocks Guava upgrading. HIVE-27560[3] aim to make Hive 2.3.10(unreleased)
compatible with all Guava 14+ versions, but unfortunately, Guava is marked as
isSharedClass[3] in IsolatedClientLoader, so technically, if we want to
upgrade Guava we need to make all supported Hive versions(through 2.0.x to
3.1.x) to support high version of Guava, I think it's impossible.

[1] sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientVersions.scala
[2] https://issues.apache.org/jira/browse/SPARK-42539
[3] https://issues.apache.org/jira/browse/HIVE-27560
[4] #33989 (comment)

sunchao · 2023-08-16T01:40:16Z

Dropping IsolatedClientLoader is also my main concern here - I'm not sure whether anyone in the community still rely on the feature for some reason (cc @xkrogen). I do agree that Hive 2.3.9/10 should be compatible with most Hive versions.

Perhaps it's worth checking out in the dev mailing list, if it's not done already.

pan3793 · 2023-08-16T04:15:24Z

Perhaps it's worth checking out in the dev mailing list, if it's not done already.

@sunchao I posted the proposal in the mailing list thread What else could be removed in Spark 4?.

pan3793 · 2023-08-16T04:34:34Z

I do agree that Hive 2.3.9/10 should be compatible with most Hive versions.

Yes, I can contribute some use cases.

We use Hive 2.3.9 to access HMS 2.1/2.3 internally for several years.
In default setup of E-MapReduce of AliCloud, it uses Spark 3.3.1(Hive 2.3.9) to access HMS 3.1.

xkrogen · 2023-08-16T17:49:45Z

Dropping IsolatedClientLoader is also my main concern here - I'm not sure whether anyone in the community still rely on the feature for some reason (cc @xkrogen). I do agree that Hive 2.3.9/10 should be compatible with most Hive versions.

Thanks for the ping @sunchao! We're still using it in Spark 3.1.1 to access Hive 1.1, but we're working to get rid of it after the compatibility work you did on the HMS side (thanks for that!). Our only blocker has been some rollout concerns, nothing from the compatibility angle. So, good from my perspective.

cxzl25 · 2023-08-21T08:19:40Z

Sharing a real case from my company.
Using Spark version 3.2 in production environment, using buildin's Hive Client version 2.3.9 with two patches, communicating with Hive meta store 1.1.0, it has been working fine for three years now.

HIVE-25500 Switch back to alter_partition(s) in HMS client for Hive 2.3.x
[SPARK-44454][SQL][HIVE] HiveShim getTablesByType support fallback

JoshRosen · 2023-08-21T21:50:16Z

Regarding the proposed removal of IsolatedClientLoader, I am concerned that doing so might break some users' ability to interface with very old Hive metastores:

Although newer Hive client versions are compatible with a wider range of metastore versions, I am unsure whether they cover all old versions. If users are relying on Hive 0.13, for example, then I am concerned that removing IsolatedClientLoader might prevent those users from being able to upgrade Spark without first upgrading their metastore.

As was pointed out upthread in #42493 (comment), though, those old Hive versions require Java 8. If Spark 4.0 drops JDK8 support then we'd already lose support for Hive < 2.0 using out-of-the-box Hive metastore clients.

However, some users run forked versions of old Hive which do support newer JDKs and for them the IsolatedClientLoader removal would create pains for their Spark 4.0 upgrade. Since those users are already forking Hive, though, I suppose they could simply recompile their Hive fork against Spark's newer Guava version.

That said, I wonder whether we can decouple the IsolatedClientLoader removal decision from the Guava upgrade by removing Guava from sharedClasses (previously discussed at #33989 (comment)). This could also prove useful in case we uncover some other reason why we might want to continue using IsolatedClientLoader.

I think Guava would only need to be shared if we were passing instances of Guava classes across the classloader boundary, but I don't think we're doing that.

I tried removing Guava from sharedClasses and used SBT to run org.apache.spark.sql.hive.client.HiveClientSuites and tests seem to pass. Given this, maybe we could try kicking off a full CI run to see if we get any failures there? I think it may be worth re-testing here in order to verify our assumptions about Guava and the IsolatedClientLoader.

pan3793 · 2023-08-22T06:44:09Z

@JoshRosen thanks for the comments, #42599 is opened to try removing Guava from sharedClasses, let's wait for CI results.

If Spark 4.0 drops JDK8 support then we'd already lose support for Hive < 2.0 using out-of-the-box Hive metastore clients.

Yes, definitely, from the community perspective, that means Spark 4.0 drops support for HMS < 2.0.

However, some users run forked versions of old Hive which do support newer JDKs and for them the IsolatedClientLoader removal would create pains for their Spark 4.0 upgrade. Since those users are already forking Hive, though, I suppose they could simply recompile their Hive fork against Spark's newer Guava version.

I think it's out of the scope of the Spark community's responsibility. And it's very weird to combine Hive 0.13(released in 2014) with Spark 4.0(will release in 2024), considering that Spark 4.0 decided to drop support for Java 8 and Hadoop 2, I don't think we should insist to support such an old Hive version.

I definitely see lots of users fork and maintain their internal Hadoop/Hive/Spark versions, but as you said, they could simply recompile their Hive fork against Spark's newer Guava version, I suppose they also could recompile their Spark fork against their Hive fork.

In short, I think it does not make sense that a user expects Spark 4.0.0 should work out-of-box with a forked Hive released a decade ago. He/she should expect some code changes on their Spark fork to make such a combination work.

github-actions · 2023-12-01T00:21:08Z

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

pan3793 · 2023-12-18T03:28:49Z

UPDATE

Dropping IsolatedClientLoader is not necessary after we dropped support for Hive prior 2.0.0 in SPARK-45328, see technical details at #42599.

I think there are no known technical issues blocking us from upgrading Guava while keeping IsolatedClientLoader in Spark 4.0.0

Summary of upgrading Guava steps:

SPARK-45292: Remove Guava from shared classes from IsolatedClientLoader [SPARK-45292][SQL][HIVE] Remove Guava from shared classes from IsolatedClientLoader #42599
Wait for Hive 2.3.10 for HIVE-27560
Upgrade Spark built-in Guava to 32+

…edClientLoader ### What changes were proposed in this pull request? Try removing Guava from `sharedClasses` as suggested by JoshRosen in https://github.com/apache/spark/pull/33989#issuecomment-928616327 and https://github.com/apache/spark/pull/42493#issuecomment-1687092403 ### Why are the changes needed? Unblock Guava upgrading. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI passed (embedded HMS) and verified in the internal YARN cluster (remote HMS with kerberos-enabled). ``` # already setup hive-site.xml stuff properly to make sure to use remote HMS bin/spark-shell --conf spark.sql.hive.metastore.jars=maven ... scala> spark.sql("show databases").show warning: 1 deprecation (since 2.13.3); for details, enable `:setting -deprecation` or `:replay -deprecation` https://maven-central.storage-download.googleapis.com/maven2/ added as a remote repository with the name: repo-1 Ivy Default Cache set to: /home/hadoop/.ivy2/cache The jars for the packages stored in: /home/hadoop/.ivy2/jars org.apache.hive#hive-metastore added as a dependency org.apache.hive#hive-exec added as a dependency org.apache.hive#hive-common added as a dependency org.apache.hive#hive-serde added as a dependency org.apache.hadoop#hadoop-client-api added as a dependency org.apache.hadoop#hadoop-client-runtime added as a dependency :: resolving dependencies :: org.apache.spark#spark-submit-parent-d0d2962d-ae27-4526-a0c7-040a542e1e54;1.0 confs: [default] found org.apache.hive#hive-metastore;2.3.9 in central found org.apache.hive#hive-serde;2.3.9 in central found org.apache.hive#hive-common;2.3.9 in central found org.apache.hive#hive-shims;2.3.9 in central found org.apache.hive.shims#hive-shims-common;2.3.9 in central found org.apache.logging.log4j#log4j-slf4j-impl;2.6.2 in central found org.slf4j#slf4j-api;1.7.10 in central found com.google.guava#guava;14.0.1 in central found commons-lang#commons-lang;2.6 in central found org.apache.thrift#libthrift;0.9.3 in central found org.apache.httpcomponents#httpclient;4.4 in central found org.apache.httpcomponents#httpcore;4.4 in central found commons-logging#commons-logging;1.2 in central found commons-codec#commons-codec;1.4 in central found org.apache.zookeeper#zookeeper;3.4.6 in central found org.slf4j#slf4j-log4j12;1.6.1 in central found log4j#log4j;1.2.16 in central found jline#jline;2.12 in central found io.netty#netty;3.7.0.Final in central found org.apache.hive.shims#hive-shims-0.23;2.3.9 in central found org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.7.2 in central found org.apache.hadoop#hadoop-annotations;2.7.2 in central found com.google.inject.extensions#guice-servlet;3.0 in central found com.google.inject#guice;3.0 in central found javax.inject#javax.inject;1 in central found aopalliance#aopalliance;1.0 in central found org.sonatype.sisu.inject#cglib;2.2.1-v20090111 in central found asm#asm;3.2 in central found com.google.protobuf#protobuf-java;2.5.0 in central found commons-io#commons-io;2.4 in central found com.sun.jersey#jersey-json;1.14 in central found org.codehaus.jettison#jettison;1.1 in central found com.sun.xml.bind#jaxb-impl;2.2.3-1 in central found javax.xml.bind#jaxb-api;2.2.2 in central found javax.xml.stream#stax-api;1.0-2 in central found javax.activation#activation;1.1 in central found org.codehaus.jackson#jackson-core-asl;1.9.13 in central found org.codehaus.jackson#jackson-mapper-asl;1.9.13 in central found org.codehaus.jackson#jackson-jaxrs;1.9.13 in central found org.codehaus.jackson#jackson-xc;1.9.13 in central found com.sun.jersey#jersey-core;1.14 in central found com.sun.jersey.contribs#jersey-guice;1.9 in central found com.sun.jersey#jersey-server;1.14 in central found org.apache.hadoop#hadoop-yarn-common;2.7.2 in central found org.apache.hadoop#hadoop-yarn-api;2.7.2 in central found org.apache.commons#commons-compress;1.9 in central found org.mortbay.jetty#jetty-util;6.1.26 in central found com.sun.jersey#jersey-client;1.9 in central found commons-cli#commons-cli;1.2 in central found log4j#log4j;1.2.17 in central found org.apache.hadoop#hadoop-yarn-server-common;2.7.2 in central found org.fusesource.leveldbjni#leveldbjni-all;1.8 in central found org.apache.hadoop#hadoop-yarn-server-applicationhistoryservice;2.7.2 in central found commons-collections#commons-collections;3.2.2 in central found org.apache.hadoop#hadoop-yarn-server-web-proxy;2.7.2 in central found org.mortbay.jetty#jetty;6.1.26 in central found org.apache.hive.shims#hive-shims-scheduler;2.3.9 in central found org.apache.hive#hive-storage-api;2.4.0 in central found org.apache.commons#commons-lang3;3.1 in central found org.apache.orc#orc-core;1.3.4 in central found io.airlift#aircompressor;0.8 in central found io.airlift#slice;0.29 in central found org.openjdk.jol#jol-core;0.2 in central found org.eclipse.jetty.aggregate#jetty-all;7.6.0.v20120127 in central found org.apache.geronimo.specs#geronimo-jta_1.1_spec;1.1.1 in central found javax.mail#mail;1.4.1 in central found org.apache.geronimo.specs#geronimo-jaspic_1.0_spec;1.0 in central found org.apache.geronimo.specs#geronimo-annotation_1.0_spec;1.1.1 in central found asm#asm-commons;3.1 in central found asm#asm-tree;3.1 in central found org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016 in central found joda-time#joda-time;2.8.1 in central found org.apache.logging.log4j#log4j-1.2-api;2.6.2 in central found org.apache.logging.log4j#log4j-web;2.6.2 in central found org.apache.ant#ant;1.9.1 in central found org.apache.ant#ant-launcher;1.9.1 in central found com.tdunning#json;1.8 in central found io.dropwizard.metrics#metrics-core;3.1.0 in central found io.dropwizard.metrics#metrics-jvm;3.1.0 in central found io.dropwizard.metrics#metrics-json;3.1.0 in central found com.github.joshelser#dropwizard-metrics-hadoop-metrics2-reporter;0.1.2 in central found org.apache.hadoop#hadoop-common;2.7.2 in central found org.apache.commons#commons-math3;3.1.1 in central found xmlenc#xmlenc;0.52 in central found commons-httpclient#commons-httpclient;3.1 in central found commons-net#commons-net;3.1 in central found javax.servlet#servlet-api;2.5 in central found net.java.dev.jets3t#jets3t;0.9.0 in central found com.jamesmurty.utils#java-xmlbuilder;0.4 in central found commons-configuration#commons-configuration;1.6 in central found commons-digester#commons-digester;1.8 in central found commons-beanutils#commons-beanutils;1.7.0 in central found commons-beanutils#commons-beanutils-core;1.8.0 in central found org.apache.avro#avro;1.8.2 in central found com.thoughtworks.paranamer#paranamer;2.7 in central found org.xerial.snappy#snappy-java;1.1.1.3 in central found org.tukaani#xz;1.5 in central found com.google.code.gson#gson;2.2.4 in central found org.apache.hadoop#hadoop-auth;2.7.2 in central found org.apache.directory.server#apacheds-kerberos-codec;2.0.0-M15 in central found org.apache.directory.server#apacheds-i18n;2.0.0-M15 in central found org.apache.directory.api#api-asn1-api;1.0.0-M20 in central found org.apache.directory.api#api-util;1.0.0-M20 in central found com.jcraft#jsch;0.1.42 in central found com.google.code.findbugs#jsr305;3.0.0 in central found org.apache.htrace#htrace-core;3.1.0-incubating in central found javax.servlet.jsp#jsp-api;2.1 in central found org.slf4j#slf4j-log4j12;1.7.14 in central found org.apache.hive#hive-service-rpc;2.3.9 in central found tomcat#jasper-compiler;5.5.23 in central found javax.servlet#jsp-api;2.0 in central found ant#ant;1.6.5 in central found tomcat#jasper-runtime;5.5.23 in central found commons-el#commons-el;1.0 in central found org.apache.thrift#libfb303;0.9.3 in central found net.sf.opencsv#opencsv;2.3 in central found org.apache.parquet#parquet-hadoop-bundle;1.8.1 in central found javolution#javolution;5.5.1 in central found org.apache.hbase#hbase-client;1.1.1 in central found org.apache.hbase#hbase-annotations;1.1.1 in central found com.github.stephenc.findbugs#findbugs-annotations;1.3.9-1 in central found junit#junit;4.11 in central found org.hamcrest#hamcrest-core;1.3 in central found org.apache.hbase#hbase-protocol;1.1.1 in central found io.netty#netty-all;4.0.52.Final in central found org.jruby.jcodings#jcodings;1.0.8 in central found org.jruby.joni#joni;2.1.2 in central found org.apache.hadoop#hadoop-mapreduce-client-core;2.7.2 in central found com.jolbox#bonecp;0.8.0.RELEASE in central found com.zaxxer#HikariCP;2.5.1 in central found org.apache.derby#derby;10.10.2.0 in central found org.datanucleus#datanucleus-api-jdo;4.2.4 in central found org.datanucleus#datanucleus-core;4.1.17 in central found org.datanucleus#datanucleus-rdbms;4.1.19 in central found commons-pool#commons-pool;1.5.4 in central found commons-dbcp#commons-dbcp;1.4 in central found javax.jdo#jdo-api;3.0.1 in central found javax.transaction#jta;1.1 in central found org.datanucleus#javax.jdo;3.2.0-m3 in central found javax.transaction#transaction-api;1.1 in central found org.antlr#antlr-runtime;3.5.2 in central found co.cask.tephra#tephra-api;0.6.0 in central found co.cask.tephra#tephra-core;0.6.0 in central found com.google.inject.extensions#guice-assistedinject;3.0 in central found it.unimi.dsi#fastutil;6.5.6 in central found org.apache.twill#twill-common;0.6.0-incubating in central found org.apache.twill#twill-core;0.6.0-incubating in central found org.apache.twill#twill-api;0.6.0-incubating in central found org.apache.twill#twill-discovery-api;0.6.0-incubating in central found org.apache.twill#twill-zookeeper;0.6.0-incubating in central found org.apache.twill#twill-discovery-core;0.6.0-incubating in central found co.cask.tephra#tephra-hbase-compat-1.0;0.6.0 in central found org.apache.hive#hive-exec;2.3.9 in central found org.apache.hive#hive-llap-tez;2.3.9 in central found org.apache.hive#hive-llap-client;2.3.9 in central found org.apache.hive#hive-llap-common;2.3.9 in central found org.antlr#ST4;4.0.4 in central found org.apache.ivy#ivy;2.4.0 in central found org.codehaus.groovy#groovy-all;2.4.4 in central found stax#stax-api;1.0.1 in central found net.hydromatic#eigenbase-properties;1.1.5 in central found org.codehaus.janino#commons-compiler;2.7.6 in central found org.codehaus.janino#janino;2.7.6 in central found org.apache.hadoop#hadoop-client-api;3.3.6 in central found org.xerial.snappy#snappy-java;1.1.8.2 in central found org.apache.hadoop#hadoop-client-runtime;3.3.6 in central found org.slf4j#slf4j-api;1.7.36 in central found com.google.code.findbugs#jsr305;3.0.2 in central downloading https://repo1.maven.org/maven2/org/apache/hive/hive-metastore/2.3.9/hive-metastore-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-metastore;2.3.9!hive-metastore.jar (1292ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-exec/2.3.9/hive-exec-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-exec;2.3.9!hive-exec.jar (1769ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-common/2.3.9/hive-common-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-common;2.3.9!hive-common.jar (360ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-serde/2.3.9/hive-serde-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-serde;2.3.9!hive-serde.jar (370ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client-api/3.3.6/hadoop-client-api-3.3.6.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-client-api;3.3.6!hadoop-client-api.jar (936ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client-runtime/3.3.6/hadoop-client-runtime-3.3.6.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-client-runtime;3.3.6!hadoop-client-runtime.jar (1418ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-shims/2.3.9/hive-shims-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-shims;2.3.9!hive-shims.jar (361ms) downloading https://repo1.maven.org/maven2/javolution/javolution/5.5.1/javolution-5.5.1.jar ... [SUCCESSFUL ] javolution#javolution;5.5.1!javolution.jar(bundle) (359ms) downloading https://repo1.maven.org/maven2/com/google/guava/guava/14.0.1/guava-14.0.1.jar ... [SUCCESSFUL ] com.google.guava#guava;14.0.1!guava.jar(bundle) (385ms) downloading https://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar ... [SUCCESSFUL ] com.google.protobuf#protobuf-java;2.5.0!protobuf-java.jar(bundle) (362ms) downloading https://repo1.maven.org/maven2/org/apache/hbase/hbase-client/1.1.1/hbase-client-1.1.1.jar ... [SUCCESSFUL ] org.apache.hbase#hbase-client;1.1.1!hbase-client.jar (378ms) downloading https://repo1.maven.org/maven2/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar ... [SUCCESSFUL ] com.jolbox#bonecp;0.8.0.RELEASE!bonecp.jar(bundle) (356ms) downloading https://repo1.maven.org/maven2/com/zaxxer/HikariCP/2.5.1/HikariCP-2.5.1.jar ... [SUCCESSFUL ] com.zaxxer#HikariCP;2.5.1!HikariCP.jar(bundle) (356ms) downloading https://repo1.maven.org/maven2/commons-cli/commons-cli/1.2/commons-cli-1.2.jar ... [SUCCESSFUL ] commons-cli#commons-cli;1.2!commons-cli.jar (353ms) downloading https://repo1.maven.org/maven2/commons-lang/commons-lang/2.6/commons-lang-2.6.jar ... [SUCCESSFUL ] commons-lang#commons-lang;2.6!commons-lang.jar (372ms) downloading https://repo1.maven.org/maven2/org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar ... [SUCCESSFUL ] org.apache.derby#derby;10.10.2.0!derby.jar (398ms) downloading https://repo1.maven.org/maven2/org/datanucleus/datanucleus-api-jdo/4.2.4/datanucleus-api-jdo-4.2.4.jar ... [SUCCESSFUL ] org.datanucleus#datanucleus-api-jdo;4.2.4!datanucleus-api-jdo.jar (361ms) downloading https://repo1.maven.org/maven2/org/datanucleus/datanucleus-core/4.1.17/datanucleus-core-4.1.17.jar ... [SUCCESSFUL ] org.datanucleus#datanucleus-core;4.1.17!datanucleus-core.jar (391ms) downloading https://repo1.maven.org/maven2/org/datanucleus/datanucleus-rdbms/4.1.19/datanucleus-rdbms-4.1.19.jar ... [SUCCESSFUL ] org.datanucleus#datanucleus-rdbms;4.1.19!datanucleus-rdbms.jar (384ms) downloading https://repo1.maven.org/maven2/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar ... [SUCCESSFUL ] commons-pool#commons-pool;1.5.4!commons-pool.jar (354ms) downloading https://repo1.maven.org/maven2/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar ... [SUCCESSFUL ] commons-dbcp#commons-dbcp;1.4!commons-dbcp.jar (355ms) downloading https://repo1.maven.org/maven2/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar ... [SUCCESSFUL ] javax.jdo#jdo-api;3.0.1!jdo-api.jar (357ms) downloading https://repo1.maven.org/maven2/org/datanucleus/javax.jdo/3.2.0-m3/javax.jdo-3.2.0-m3.jar ... [SUCCESSFUL ] org.datanucleus#javax.jdo;3.2.0-m3!javax.jdo.jar (357ms) downloading https://repo1.maven.org/maven2/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar ... [SUCCESSFUL ] org.antlr#antlr-runtime;3.5.2!antlr-runtime.jar (355ms) downloading https://repo1.maven.org/maven2/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar ... [SUCCESSFUL ] org.apache.thrift#libfb303;0.9.3!libfb303.jar (181ms) downloading https://repo1.maven.org/maven2/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar ... [SUCCESSFUL ] org.apache.thrift#libthrift;0.9.3!libthrift.jar (183ms) downloading https://repo1.maven.org/maven2/co/cask/tephra/tephra-api/0.6.0/tephra-api-0.6.0.jar ... [SUCCESSFUL ] co.cask.tephra#tephra-api;0.6.0!tephra-api.jar (353ms) downloading https://repo1.maven.org/maven2/co/cask/tephra/tephra-core/0.6.0/tephra-core-0.6.0.jar ... [SUCCESSFUL ] co.cask.tephra#tephra-core;0.6.0!tephra-core.jar (363ms) downloading https://repo1.maven.org/maven2/co/cask/tephra/tephra-hbase-compat-1.0/0.6.0/tephra-hbase-compat-1.0-0.6.0.jar ... [SUCCESSFUL ] co.cask.tephra#tephra-hbase-compat-1.0;0.6.0!tephra-hbase-compat-1.0.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-service-rpc/2.3.9/hive-service-rpc-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-service-rpc;2.3.9!hive-service-rpc.jar (379ms) downloading https://repo1.maven.org/maven2/commons-codec/commons-codec/1.4/commons-codec-1.4.jar ... [SUCCESSFUL ] commons-codec#commons-codec;1.4!commons-codec.jar (355ms) downloading https://repo1.maven.org/maven2/org/apache/avro/avro/1.8.2/avro-1.8.2.jar ... [SUCCESSFUL ] org.apache.avro#avro;1.8.2!avro.jar(bundle) (392ms) downloading https://repo1.maven.org/maven2/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar ... [SUCCESSFUL ] net.sf.opencsv#opencsv;2.3!opencsv.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/parquet/parquet-hadoop-bundle/1.8.1/parquet-hadoop-bundle-1.8.1.jar ... [SUCCESSFUL ] org.apache.parquet#parquet-hadoop-bundle;1.8.1!parquet-hadoop-bundle.jar (408ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-storage-api/2.4.0/hive-storage-api-2.4.0.jar ... [SUCCESSFUL ] org.apache.hive#hive-storage-api;2.4.0!hive-storage-api.jar (356ms) downloading https://repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.1/commons-lang3-3.1.jar ... [SUCCESSFUL ] org.apache.commons#commons-lang3;3.1!commons-lang3.jar (358ms) downloading https://repo1.maven.org/maven2/org/apache/orc/orc-core/1.3.4/orc-core-1.3.4.jar ... [SUCCESSFUL ] org.apache.orc#orc-core;1.3.4!orc-core.jar (391ms) downloading https://repo1.maven.org/maven2/jline/jline/2.12/jline-2.12.jar ... [SUCCESSFUL ] jline#jline;2.12!jline.jar (357ms) downloading https://repo1.maven.org/maven2/org/eclipse/jetty/aggregate/jetty-all/7.6.0.v20120127/jetty-all-7.6.0.v20120127.jar ... [SUCCESSFUL ] org.eclipse.jetty.aggregate#jetty-all;7.6.0.v20120127!jetty-all.jar (379ms) downloading https://repo1.maven.org/maven2/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar ... [SUCCESSFUL ] org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016!javax.servlet.jar(orbit) (359ms) downloading https://repo1.maven.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar ... [SUCCESSFUL ] joda-time#joda-time;2.8.1!joda-time.jar (361ms) downloading https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-1.2-api/2.6.2/log4j-1.2-api-2.6.2.jar ... [SUCCESSFUL ] org.apache.logging.log4j#log4j-1.2-api;2.6.2!log4j-1.2-api.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-web/2.6.2/log4j-web-2.6.2.jar ... [SUCCESSFUL ] org.apache.logging.log4j#log4j-web;2.6.2!log4j-web.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-slf4j-impl/2.6.2/log4j-slf4j-impl-2.6.2.jar ... [SUCCESSFUL ] org.apache.logging.log4j#log4j-slf4j-impl;2.6.2!log4j-slf4j-impl.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/commons/commons-compress/1.9/commons-compress-1.9.jar ... [SUCCESSFUL ] org.apache.commons#commons-compress;1.9!commons-compress.jar (359ms) downloading https://repo1.maven.org/maven2/org/apache/ant/ant/1.9.1/ant-1.9.1.jar ... [SUCCESSFUL ] org.apache.ant#ant;1.9.1!ant.jar (390ms) downloading https://repo1.maven.org/maven2/com/tdunning/json/1.8/json-1.8.jar ... [SUCCESSFUL ] com.tdunning#json;1.8!json.jar (353ms) downloading https://repo1.maven.org/maven2/io/dropwizard/metrics/metrics-core/3.1.0/metrics-core-3.1.0.jar ... [SUCCESSFUL ] io.dropwizard.metrics#metrics-core;3.1.0!metrics-core.jar(bundle) (356ms) downloading https://repo1.maven.org/maven2/io/dropwizard/metrics/metrics-jvm/3.1.0/metrics-jvm-3.1.0.jar ... [SUCCESSFUL ] io.dropwizard.metrics#metrics-jvm;3.1.0!metrics-jvm.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/io/dropwizard/metrics/metrics-json/3.1.0/metrics-json-3.1.0.jar ... [SUCCESSFUL ] io.dropwizard.metrics#metrics-json;3.1.0!metrics-json.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/com/github/joshelser/dropwizard-metrics-hadoop-metrics2-reporter/0.1.2/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar ... [SUCCESSFUL ] com.github.joshelser#dropwizard-metrics-hadoop-metrics2-reporter;0.1.2!dropwizard-metrics-hadoop-metrics2-reporter.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/hive/shims/hive-shims-common/2.3.9/hive-shims-common-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive.shims#hive-shims-common;2.3.9!hive-shims-common.jar (355ms) downloading https://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6-tests.jar ... [SUCCESSFUL ] org.apache.zookeeper#zookeeper;3.4.6!zookeeper.jar(test-jar) (362ms) downloading https://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar ... [SUCCESSFUL ] org.apache.zookeeper#zookeeper;3.4.6!zookeeper.jar (188ms) downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.4/httpclient-4.4.jar ... [SUCCESSFUL ] org.apache.httpcomponents#httpclient;4.4!httpclient.jar (368ms) downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4/httpcore-4.4.jar ... [SUCCESSFUL ] org.apache.httpcomponents#httpcore;4.4!httpcore.jar (359ms) downloading https://repo1.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar ... [SUCCESSFUL ] commons-logging#commons-logging;1.2!commons-logging.jar (353ms) downloading https://repo1.maven.org/maven2/io/netty/netty/3.7.0.Final/netty-3.7.0.Final.jar ... [SUCCESSFUL ] io.netty#netty;3.7.0.Final!netty.jar(bundle) (373ms) downloading https://repo1.maven.org/maven2/org/apache/hive/shims/hive-shims-0.23/2.3.9/hive-shims-0.23-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive.shims#hive-shims-0.23;2.3.9!hive-shims-0.23.jar (359ms) downloading https://repo1.maven.org/maven2/org/apache/hive/shims/hive-shims-scheduler/2.3.9/hive-shims-scheduler-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive.shims#hive-shims-scheduler;2.3.9!hive-shims-scheduler.jar (355ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-server-resourcemanager/2.7.2/hadoop-yarn-server-resourcemanager-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.7.2!hadoop-yarn-server-resourcemanager.jar (375ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-annotations/2.7.2/hadoop-annotations-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-annotations;2.7.2!hadoop-annotations.jar (361ms) downloading https://repo1.maven.org/maven2/com/google/inject/extensions/guice-servlet/3.0/guice-servlet-3.0.jar ... [SUCCESSFUL ] com.google.inject.extensions#guice-servlet;3.0!guice-servlet.jar (355ms) downloading https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar ... [SUCCESSFUL ] commons-io#commons-io;2.4!commons-io.jar (357ms) downloading https://repo1.maven.org/maven2/com/google/inject/guice/3.0/guice-3.0.jar ... [SUCCESSFUL ] com.google.inject#guice;3.0!guice.jar (368ms) downloading https://repo1.maven.org/maven2/com/sun/jersey/jersey-json/1.14/jersey-json-1.14.jar ... [SUCCESSFUL ] com.sun.jersey#jersey-json;1.14!jersey-json.jar (356ms) downloading https://repo1.maven.org/maven2/com/sun/jersey/contribs/jersey-guice/1.9/jersey-guice-1.9.jar ... [SUCCESSFUL ] com.sun.jersey.contribs#jersey-guice;1.9!jersey-guice.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-common/2.7.2/hadoop-yarn-common-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-yarn-common;2.7.2!hadoop-yarn-common.jar (379ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.7.2/hadoop-yarn-api-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-yarn-api;2.7.2!hadoop-yarn-api.jar (386ms) downloading https://repo1.maven.org/maven2/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar ... [SUCCESSFUL ] javax.xml.bind#jaxb-api;2.2.2!jaxb-api.jar (356ms) downloading https://repo1.maven.org/maven2/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar ... [SUCCESSFUL ] org.codehaus.jettison#jettison;1.1!jettison.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/com/sun/jersey/jersey-core/1.14/jersey-core-1.14.jar ... [SUCCESSFUL ] com.sun.jersey#jersey-core;1.14!jersey-core.jar (360ms) downloading https://repo1.maven.org/maven2/com/sun/jersey/jersey-client/1.9/jersey-client-1.9.jar ... [SUCCESSFUL ] com.sun.jersey#jersey-client;1.9!jersey-client.jar(bundle) (355ms) downloading https://repo1.maven.org/maven2/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar ... [SUCCESSFUL ] org.mortbay.jetty#jetty-util;6.1.26!jetty-util.jar (355ms) downloading https://repo1.maven.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar ... [SUCCESSFUL ] log4j#log4j;1.2.17!log4j.jar(bundle) (361ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.7.2/hadoop-yarn-server-common-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-yarn-server-common;2.7.2!hadoop-yarn-server-common.jar (367ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-server-applicationhistoryservice/2.7.2/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-yarn-server-applicationhistoryservice;2.7.2!hadoop-yarn-server-applicationhistoryservice.jar (363ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-server-web-proxy/2.7.2/hadoop-yarn-server-web-proxy-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-yarn-server-web-proxy;2.7.2!hadoop-yarn-server-web-proxy.jar (354ms) downloading https://repo1.maven.org/maven2/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar ... [SUCCESSFUL ] org.fusesource.leveldbjni#leveldbjni-all;1.8!leveldbjni-all.jar(bundle) (372ms) downloading https://repo1.maven.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar ... [SUCCESSFUL ] javax.inject#javax.inject;1!javax.inject.jar (355ms) downloading https://repo1.maven.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.jar ... [SUCCESSFUL ] aopalliance#aopalliance;1.0!aopalliance.jar (368ms) downloading https://repo1.maven.org/maven2/org/sonatype/sisu/inject/cglib/2.2.1-v20090111/cglib-2.2.1-v20090111.jar ... [SUCCESSFUL ] org.sonatype.sisu.inject#cglib;2.2.1-v20090111!cglib.jar (384ms) downloading https://repo1.maven.org/maven2/asm/asm/3.2/asm-3.2.jar ... [SUCCESSFUL ] asm#asm;3.2!asm.jar (355ms) downloading https://repo1.maven.org/maven2/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar ... [SUCCESSFUL ] com.sun.xml.bind#jaxb-impl;2.2.3-1!jaxb-impl.jar (367ms) downloading https://repo1.maven.org/maven2/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar ... [SUCCESSFUL ] org.codehaus.jackson#jackson-core-asl;1.9.13!jackson-core-asl.jar (357ms) downloading https://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar ... [SUCCESSFUL ] org.codehaus.jackson#jackson-mapper-asl;1.9.13!jackson-mapper-asl.jar (365ms) downloading https://repo1.maven.org/maven2/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar ... [SUCCESSFUL ] org.codehaus.jackson#jackson-jaxrs;1.9.13!jackson-jaxrs.jar (356ms) downloading https://repo1.maven.org/maven2/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar ... [SUCCESSFUL ] org.codehaus.jackson#jackson-xc;1.9.13!jackson-xc.jar (354ms) downloading https://repo1.maven.org/maven2/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar ... [SUCCESSFUL ] javax.xml.stream#stax-api;1.0-2!stax-api.jar (354ms) downloading https://repo1.maven.org/maven2/javax/activation/activation/1.1/activation-1.1.jar ... [SUCCESSFUL ] javax.activation#activation;1.1!activation.jar (354ms) downloading https://repo1.maven.org/maven2/com/sun/jersey/jersey-server/1.14/jersey-server-1.14.jar ... [SUCCESSFUL ] com.sun.jersey#jersey-server;1.14!jersey-server.jar (362ms) downloading https://repo1.maven.org/maven2/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar ... [SUCCESSFUL ] commons-collections#commons-collections;3.2.2!commons-collections.jar (366ms) downloading https://repo1.maven.org/maven2/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar ... [SUCCESSFUL ] org.mortbay.jetty#jetty;6.1.26!jetty.jar (362ms) downloading https://repo1.maven.org/maven2/io/airlift/aircompressor/0.8/aircompressor-0.8.jar ... [SUCCESSFUL ] io.airlift#aircompressor;0.8!aircompressor.jar (358ms) downloading https://repo1.maven.org/maven2/io/airlift/slice/0.29/slice-0.29.jar ... [SUCCESSFUL ] io.airlift#slice;0.29!slice.jar (355ms) downloading https://repo1.maven.org/maven2/org/openjdk/jol/jol-core/0.2/jol-core-0.2.jar ... [SUCCESSFUL ] org.openjdk.jol#jol-core;0.2!jol-core.jar (357ms) downloading https://repo1.maven.org/maven2/org/apache/geronimo/specs/geronimo-jta_1.1_spec/1.1.1/geronimo-jta_1.1_spec-1.1.1.jar ... [SUCCESSFUL ] org.apache.geronimo.specs#geronimo-jta_1.1_spec;1.1.1!geronimo-jta_1.1_spec.jar (354ms) downloading https://repo1.maven.org/maven2/javax/mail/mail/1.4.1/mail-1.4.1.jar ... [SUCCESSFUL ] javax.mail#mail;1.4.1!mail.jar (361ms) downloading https://repo1.maven.org/maven2/org/apache/geronimo/specs/geronimo-jaspic_1.0_spec/1.0/geronimo-jaspic_1.0_spec-1.0.jar ... [SUCCESSFUL ] org.apache.geronimo.specs#geronimo-jaspic_1.0_spec;1.0!geronimo-jaspic_1.0_spec.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/org/apache/geronimo/specs/geronimo-annotation_1.0_spec/1.1.1/geronimo-annotation_1.0_spec-1.1.1.jar ... [SUCCESSFUL ] org.apache.geronimo.specs#geronimo-annotation_1.0_spec;1.1.1!geronimo-annotation_1.0_spec.jar (354ms) downloading https://repo1.maven.org/maven2/asm/asm-commons/3.1/asm-commons-3.1.jar ... [SUCCESSFUL ] asm#asm-commons;3.1!asm-commons.jar (354ms) downloading https://repo1.maven.org/maven2/asm/asm-tree/3.1/asm-tree-3.1.jar ... [SUCCESSFUL ] asm#asm-tree;3.1!asm-tree.jar (356ms) downloading https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.1/ant-launcher-1.9.1.jar ... [SUCCESSFUL ] org.apache.ant#ant-launcher;1.9.1!ant-launcher.jar (353ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/2.7.2/hadoop-common-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-common;2.7.2!hadoop-common.jar (424ms) downloading https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar ... [SUCCESSFUL ] org.apache.commons#commons-math3;3.1.1!commons-math3.jar (388ms) downloading https://repo1.maven.org/maven2/xmlenc/xmlenc/0.52/xmlenc-0.52.jar ... [SUCCESSFUL ] xmlenc#xmlenc;0.52!xmlenc.jar (354ms) downloading https://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar ... [SUCCESSFUL ] commons-httpclient#commons-httpclient;3.1!commons-httpclient.jar (363ms) downloading https://repo1.maven.org/maven2/commons-net/commons-net/3.1/commons-net-3.1.jar ... [SUCCESSFUL ] commons-net#commons-net;3.1!commons-net.jar (360ms) downloading https://repo1.maven.org/maven2/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar ... [SUCCESSFUL ] javax.servlet#servlet-api;2.5!servlet-api.jar (360ms) downloading https://repo1.maven.org/maven2/net/java/dev/jets3t/jets3t/0.9.0/jets3t-0.9.0.jar ... [SUCCESSFUL ] net.java.dev.jets3t#jets3t;0.9.0!jets3t.jar (363ms) downloading https://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar ... [SUCCESSFUL ] commons-configuration#commons-configuration;1.6!commons-configuration.jar (366ms) downloading https://repo1.maven.org/maven2/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar ... [SUCCESSFUL ] com.google.code.gson#gson;2.2.4!gson.jar (367ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/2.7.2/hadoop-auth-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-auth;2.7.2!hadoop-auth.jar (360ms) downloading https://repo1.maven.org/maven2/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar ... [SUCCESSFUL ] com.jcraft#jsch;0.1.42!jsch.jar (360ms) downloading https://repo1.maven.org/maven2/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar ... [SUCCESSFUL ] org.apache.htrace#htrace-core;3.1.0-incubating!htrace-core.jar (375ms) downloading https://repo1.maven.org/maven2/com/jamesmurty/utils/java-xmlbuilder/0.4/java-xmlbuilder-0.4.jar ... [SUCCESSFUL ] com.jamesmurty.utils#java-xmlbuilder;0.4!java-xmlbuilder.jar (354ms) downloading https://repo1.maven.org/maven2/commons-digester/commons-digester/1.8/commons-digester-1.8.jar ... [SUCCESSFUL ] commons-digester#commons-digester;1.8!commons-digester.jar (355ms) downloading https://repo1.maven.org/maven2/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar ... [SUCCESSFUL ] commons-beanutils#commons-beanutils-core;1.8.0!commons-beanutils-core.jar (357ms) downloading https://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar ... [SUCCESSFUL ] commons-beanutils#commons-beanutils;1.7.0!commons-beanutils.jar (356ms) downloading https://repo1.maven.org/maven2/com/thoughtworks/paranamer/paranamer/2.7/paranamer-2.7.jar ... [SUCCESSFUL ] com.thoughtworks.paranamer#paranamer;2.7!paranamer.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/org/tukaani/xz/1.5/xz-1.5.jar ... [SUCCESSFUL ] org.tukaani#xz;1.5!xz.jar (369ms) downloading https://repo1.maven.org/maven2/org/apache/directory/server/apacheds-kerberos-codec/2.0.0-M15/apacheds-kerberos-codec-2.0.0-M15.jar ... [SUCCESSFUL ] org.apache.directory.server#apacheds-kerberos-codec;2.0.0-M15!apacheds-kerberos-codec.jar(bundle) (364ms) downloading https://repo1.maven.org/maven2/org/apache/directory/server/apacheds-i18n/2.0.0-M15/apacheds-i18n-2.0.0-M15.jar ... [SUCCESSFUL ] org.apache.directory.server#apacheds-i18n;2.0.0-M15!apacheds-i18n.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/org/apache/directory/api/api-asn1-api/1.0.0-M20/api-asn1-api-1.0.0-M20.jar ... [SUCCESSFUL ] org.apache.directory.api#api-asn1-api;1.0.0-M20!api-asn1-api.jar(bundle) (354ms) downloading https://repo1.maven.org/maven2/org/apache/directory/api/api-util/1.0.0-M20/api-util-1.0.0-M20.jar ... [SUCCESSFUL ] org.apache.directory.api#api-util;1.0.0-M20!api-util.jar(bundle) (355ms) downloading https://repo1.maven.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar ... [SUCCESSFUL ] javax.servlet.jsp#jsp-api;2.1!jsp-api.jar (356ms) downloading https://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.7.14/slf4j-log4j12-1.7.14.jar ... [SUCCESSFUL ] org.slf4j#slf4j-log4j12;1.7.14!slf4j-log4j12.jar (354ms) downloading https://repo1.maven.org/maven2/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar ... [SUCCESSFUL ] tomcat#jasper-compiler;5.5.23!jasper-compiler.jar (360ms) downloading https://repo1.maven.org/maven2/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar ... [SUCCESSFUL ] tomcat#jasper-runtime;5.5.23!jasper-runtime.jar (355ms) downloading https://repo1.maven.org/maven2/javax/servlet/jsp-api/2.0/jsp-api-2.0.jar ... [SUCCESSFUL ] javax.servlet#jsp-api;2.0!jsp-api.jar (355ms) downloading https://repo1.maven.org/maven2/ant/ant/1.6.5/ant-1.6.5.jar ... [SUCCESSFUL ] ant#ant;1.6.5!ant.jar (371ms) downloading https://repo1.maven.org/maven2/commons-el/commons-el/1.0/commons-el-1.0.jar ... [SUCCESSFUL ] commons-el#commons-el;1.0!commons-el.jar (355ms) downloading https://repo1.maven.org/maven2/org/apache/hbase/hbase-annotations/1.1.1/hbase-annotations-1.1.1.jar ... [SUCCESSFUL ] org.apache.hbase#hbase-annotations;1.1.1!hbase-annotations.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/hbase/hbase-protocol/1.1.1/hbase-protocol-1.1.1.jar ... [SUCCESSFUL ] org.apache.hbase#hbase-protocol;1.1.1!hbase-protocol.jar (414ms) downloading https://repo1.maven.org/maven2/io/netty/netty-all/4.0.52.Final/netty-all-4.0.52.Final.jar ... [SUCCESSFUL ] io.netty#netty-all;4.0.52.Final!netty-all.jar (400ms) downloading https://repo1.maven.org/maven2/org/jruby/jcodings/jcodings/1.0.8/jcodings-1.0.8.jar ... [SUCCESSFUL ] org.jruby.jcodings#jcodings;1.0.8!jcodings.jar (419ms) downloading https://repo1.maven.org/maven2/org/jruby/joni/joni/2.1.2/joni-2.1.2.jar ... [SUCCESSFUL ] org.jruby.joni#joni;2.1.2!joni.jar (369ms) downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core/2.7.2/hadoop-mapreduce-client-core-2.7.2.jar ... [SUCCESSFUL ] org.apache.hadoop#hadoop-mapreduce-client-core;2.7.2!hadoop-mapreduce-client-core.jar (393ms) downloading https://repo1.maven.org/maven2/com/github/stephenc/findbugs/findbugs-annotations/1.3.9-1/findbugs-annotations-1.3.9-1.jar ... [SUCCESSFUL ] com.github.stephenc.findbugs#findbugs-annotations;1.3.9-1!findbugs-annotations.jar (354ms) downloading https://repo1.maven.org/maven2/junit/junit/4.11/junit-4.11.jar ... [SUCCESSFUL ] junit#junit;4.11!junit.jar (357ms) downloading https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar ... [SUCCESSFUL ] org.hamcrest#hamcrest-core;1.3!hamcrest-core.jar (353ms) downloading https://repo1.maven.org/maven2/javax/transaction/jta/1.1/jta-1.1.jar ... [SUCCESSFUL ] javax.transaction#jta;1.1!jta.jar (354ms) downloading https://repo1.maven.org/maven2/javax/transaction/transaction-api/1.1/transaction-api-1.1.jar ... [SUCCESSFUL ] javax.transaction#transaction-api;1.1!transaction-api.jar (368ms) downloading https://repo1.maven.org/maven2/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.jar ... [SUCCESSFUL ] com.google.inject.extensions#guice-assistedinject;3.0!guice-assistedinject.jar (355ms) downloading https://repo1.maven.org/maven2/it/unimi/dsi/fastutil/6.5.6/fastutil-6.5.6.jar ... [SUCCESSFUL ] it.unimi.dsi#fastutil;6.5.6!fastutil.jar (894ms) downloading https://repo1.maven.org/maven2/org/apache/twill/twill-common/0.6.0-incubating/twill-common-0.6.0-incubating.jar ... [SUCCESSFUL ] org.apache.twill#twill-common;0.6.0-incubating!twill-common.jar (355ms) downloading https://repo1.maven.org/maven2/org/apache/twill/twill-core/0.6.0-incubating/twill-core-0.6.0-incubating.jar ... [SUCCESSFUL ] org.apache.twill#twill-core;0.6.0-incubating!twill-core.jar (358ms) downloading https://repo1.maven.org/maven2/org/apache/twill/twill-discovery-api/0.6.0-incubating/twill-discovery-api-0.6.0-incubating.jar ... [SUCCESSFUL ] org.apache.twill#twill-discovery-api;0.6.0-incubating!twill-discovery-api.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/twill/twill-discovery-core/0.6.0-incubating/twill-discovery-core-0.6.0-incubating.jar ... [SUCCESSFUL ] org.apache.twill#twill-discovery-core;0.6.0-incubating!twill-discovery-core.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/twill/twill-zookeeper/0.6.0-incubating/twill-zookeeper-0.6.0-incubating.jar ... [SUCCESSFUL ] org.apache.twill#twill-zookeeper;0.6.0-incubating!twill-zookeeper.jar (357ms) downloading https://repo1.maven.org/maven2/org/apache/twill/twill-api/0.6.0-incubating/twill-api-0.6.0-incubating.jar ... [SUCCESSFUL ] org.apache.twill#twill-api;0.6.0-incubating!twill-api.jar (354ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-llap-tez/2.3.9/hive-llap-tez-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-llap-tez;2.3.9!hive-llap-tez.jar (357ms) downloading https://repo1.maven.org/maven2/org/antlr/ST4/4.0.4/ST4-4.0.4.jar ... [SUCCESSFUL ] org.antlr#ST4;4.0.4!ST4.jar (356ms) downloading https://repo1.maven.org/maven2/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar ... [SUCCESSFUL ] org.apache.ivy#ivy;2.4.0!ivy.jar (372ms) downloading https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-all/2.4.4/groovy-all-2.4.4.jar ... [SUCCESSFUL ] org.codehaus.groovy#groovy-all;2.4.4!groovy-all.jar (549ms) downloading https://repo1.maven.org/maven2/stax/stax-api/1.0.1/stax-api-1.0.1.jar ... [SUCCESSFUL ] stax#stax-api;1.0.1!stax-api.jar (354ms) downloading https://repo1.maven.org/maven2/net/hydromatic/eigenbase-properties/1.1.5/eigenbase-properties-1.1.5.jar ... [SUCCESSFUL ] net.hydromatic#eigenbase-properties;1.1.5!eigenbase-properties.jar(bundle) (356ms) downloading https://repo1.maven.org/maven2/org/codehaus/janino/commons-compiler/2.7.6/commons-compiler-2.7.6.jar ... [SUCCESSFUL ] org.codehaus.janino#commons-compiler;2.7.6!commons-compiler.jar (354ms) downloading https://repo1.maven.org/maven2/org/codehaus/janino/janino/2.7.6/janino-2.7.6.jar ... [SUCCESSFUL ] org.codehaus.janino#janino;2.7.6!janino.jar (363ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-llap-client/2.3.9/hive-llap-client-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-llap-client;2.3.9!hive-llap-client.jar (356ms) downloading https://repo1.maven.org/maven2/org/apache/hive/hive-llap-common/2.3.9/hive-llap-common-2.3.9.jar ... [SUCCESSFUL ] org.apache.hive#hive-llap-common;2.3.9!hive-llap-common.jar (358ms) downloading https://repo1.maven.org/maven2/org/xerial/snappy/snappy-java/1.1.8.2/snappy-java-1.1.8.2.jar ... [SUCCESSFUL ] org.xerial.snappy#snappy-java;1.1.8.2!snappy-java.jar(bundle) (383ms) downloading https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar ... [SUCCESSFUL ] org.slf4j#slf4j-api;1.7.36!slf4j-api.jar (353ms) downloading https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar ... [SUCCESSFUL ] com.google.code.findbugs#jsr305;3.0.2!jsr305.jar (353ms) :: resolution report :: resolve 193214ms :: artifacts dl 63795ms :: modules in use: ant#ant;1.6.5 from central in [default] aopalliance#aopalliance;1.0 from central in [default] asm#asm;3.2 from central in [default] asm#asm-commons;3.1 from central in [default] asm#asm-tree;3.1 from central in [default] co.cask.tephra#tephra-api;0.6.0 from central in [default] co.cask.tephra#tephra-core;0.6.0 from central in [default] co.cask.tephra#tephra-hbase-compat-1.0;0.6.0 from central in [default] com.github.joshelser#dropwizard-metrics-hadoop-metrics2-reporter;0.1.2 from central in [default] com.github.stephenc.findbugs#findbugs-annotations;1.3.9-1 from central in [default] com.google.code.findbugs#jsr305;3.0.2 from central in [default] com.google.code.gson#gson;2.2.4 from central in [default] com.google.guava#guava;14.0.1 from central in [default] com.google.inject#guice;3.0 from central in [default] com.google.inject.extensions#guice-assistedinject;3.0 from central in [default] com.google.inject.extensions#guice-servlet;3.0 from central in [default] com.google.protobuf#protobuf-java;2.5.0 from central in [default] com.jamesmurty.utils#java-xmlbuilder;0.4 from central in [default] com.jcraft#jsch;0.1.42 from central in [default] com.jolbox#bonecp;0.8.0.RELEASE from central in [default] com.sun.jersey#jersey-client;1.9 from central in [default] com.sun.jersey#jersey-core;1.14 from central in [default] com.sun.jersey#jersey-json;1.14 from central in [default] com.sun.jersey#jersey-server;1.14 from central in [default] com.sun.jersey.contribs#jersey-guice;1.9 from central in [default] com.sun.xml.bind#jaxb-impl;2.2.3-1 from central in [default] com.tdunning#json;1.8 from central in [default] com.thoughtworks.paranamer#paranamer;2.7 from central in [default] com.zaxxer#HikariCP;2.5.1 from central in [default] commons-beanutils#commons-beanutils;1.7.0 from central in [default] commons-beanutils#commons-beanutils-core;1.8.0 from central in [default] commons-cli#commons-cli;1.2 from central in [default] commons-codec#commons-codec;1.4 from central in [default] commons-collections#commons-collections;3.2.2 from central in [default] commons-configuration#commons-configuration;1.6 from central in [default] commons-dbcp#commons-dbcp;1.4 from central in [default] commons-digester#commons-digester;1.8 from central in [default] commons-el#commons-el;1.0 from central in [default] commons-httpclient#commons-httpclient;3.1 from central in [default] commons-io#commons-io;2.4 from central in [default] commons-lang#commons-lang;2.6 from central in [default] commons-logging#commons-logging;1.2 from central in [default] commons-net#commons-net;3.1 from central in [default] commons-pool#commons-pool;1.5.4 from central in [default] io.airlift#aircompressor;0.8 from central in [default] io.airlift#slice;0.29 from central in [default] io.dropwizard.metrics#metrics-core;3.1.0 from central in [default] io.dropwizard.metrics#metrics-json;3.1.0 from central in [default] io.dropwizard.metrics#metrics-jvm;3.1.0 from central in [default] io.netty#netty;3.7.0.Final from central in [default] io.netty#netty-all;4.0.52.Final from central in [default] it.unimi.dsi#fastutil;6.5.6 from central in [default] javax.activation#activation;1.1 from central in [default] javax.inject#javax.inject;1 from central in [default] javax.jdo#jdo-api;3.0.1 from central in [default] javax.mail#mail;1.4.1 from central in [default] javax.servlet#jsp-api;2.0 from central in [default] javax.servlet#servlet-api;2.5 from central in [default] javax.servlet.jsp#jsp-api;2.1 from central in [default] javax.transaction#jta;1.1 from central in [default] javax.transaction#transaction-api;1.1 from central in [default] javax.xml.bind#jaxb-api;2.2.2 from central in [default] javax.xml.stream#stax-api;1.0-2 from central in [default] javolution#javolution;5.5.1 from central in [default] jline#jline;2.12 from central in [default] joda-time#joda-time;2.8.1 from central in [default] junit#junit;4.11 from central in [default] log4j#log4j;1.2.17 from central in [default] net.hydromatic#eigenbase-properties;1.1.5 from central in [default] net.java.dev.jets3t#jets3t;0.9.0 from central in [default] net.sf.opencsv#opencsv;2.3 from central in [default] org.antlr#ST4;4.0.4 from central in [default] org.antlr#antlr-runtime;3.5.2 from central in [default] org.apache.ant#ant;1.9.1 from central in [default] org.apache.ant#ant-launcher;1.9.1 from central in [default] org.apache.avro#avro;1.8.2 from central in [default] org.apache.commons#commons-compress;1.9 from central in [default] org.apache.commons#commons-lang3;3.1 from central in [default] org.apache.commons#commons-math3;3.1.1 from central in [default] org.apache.derby#derby;10.10.2.0 from central in [default] org.apache.directory.api#api-asn1-api;1.0.0-M20 from central in [default] org.apache.directory.api#api-util;1.0.0-M20 from central in [default] org.apache.directory.server#apacheds-i18n;2.0.0-M15 from central in [default] org.apache.directory.server#apacheds-kerberos-codec;2.0.0-M15 from central in [default] org.apache.geronimo.specs#geronimo-annotation_1.0_spec;1.1.1 from central in [default] org.apache.geronimo.specs#geronimo-jaspic_1.0_spec;1.0 from central in [default] org.apache.geronimo.specs#geronimo-jta_1.1_spec;1.1.1 from central in [default] org.apache.hadoop#hadoop-annotations;2.7.2 from central in [default] org.apache.hadoop#hadoop-auth;2.7.2 from central in [default] org.apache.hadoop#hadoop-client-api;3.3.6 from central in [default] org.apache.hadoop#hadoop-client-runtime;3.3.6 from central in [default] org.apache.hadoop#hadoop-common;2.7.2 from central in [default] org.apache.hadoop#hadoop-mapreduce-client-core;2.7.2 from central in [default] org.apache.hadoop#hadoop-yarn-api;2.7.2 from central in [default] org.apache.hadoop#hadoop-yarn-common;2.7.2 from central in [default] org.apache.hadoop#hadoop-yarn-server-applicationhistoryservice;2.7.2 from central in [default] org.apache.hadoop#hadoop-yarn-server-common;2.7.2 from central in [default] org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.7.2 from central in [default] org.apache.hadoop#hadoop-yarn-server-web-proxy;2.7.2 from central in [default] org.apache.hbase#hbase-annotations;1.1.1 from central in [default] org.apache.hbase#hbase-client;1.1.1 from central in [default] org.apache.hbase#hbase-protocol;1.1.1 from central in [default] org.apache.hive#hive-common;2.3.9 from central in [default] org.apache.hive#hive-exec;2.3.9 from central in [default] org.apache.hive#hive-llap-client;2.3.9 from central in [default] org.apache.hive#hive-llap-common;2.3.9 from central in [default] org.apache.hive#hive-llap-tez;2.3.9 from central in [default] org.apache.hive#hive-metastore;2.3.9 from central in [default] org.apache.hive#hive-serde;2.3.9 from central in [default] org.apache.hive#hive-service-rpc;2.3.9 from central in [default] org.apache.hive#hive-shims;2.3.9 from central in [default] org.apache.hive#hive-storage-api;2.4.0 from central in [default] org.apache.hive.shims#hive-shims-0.23;2.3.9 from central in [default] org.apache.hive.shims#hive-shims-common;2.3.9 from central in [default] org.apache.hive.shims#hive-shims-scheduler;2.3.9 from central in [default] org.apache.htrace#htrace-core;3.1.0-incubating from central in [default] org.apache.httpcomponents#httpclient;4.4 from central in [default] org.apache.httpcomponents#httpcore;4.4 from central in [default] org.apache.ivy#ivy;2.4.0 from central in [default] org.apache.logging.log4j#log4j-1.2-api;2.6.2 from central in [default] org.apache.logging.log4j#log4j-slf4j-impl;2.6.2 from central in [default] org.apache.logging.log4j#log4j-web;2.6.2 from central in [default] org.apache.orc#orc-core;1.3.4 from central in [default] org.apache.parquet#parquet-hadoop-bundle;1.8.1 from central in [default] org.apache.thrift#libfb303;0.9.3 from central in [default] org.apache.thrift#libthrift;0.9.3 from central in [default] org.apache.twill#twill-api;0.6.0-incubating from central in [default] org.apache.twill#twill-common;0.6.0-incubating from central in [default] org.apache.twill#twill-core;0.6.0-incubating from central in [default] org.apache.twill#twill-discovery-api;0.6.0-incubating from central in [default] org.apache.twill#twill-discovery-core;0.6.0-incubating from central in [default] org.apache.twill#twill-zookeeper;0.6.0-incubating from central in [default] org.apache.zookeeper#zookeeper;3.4.6 from central in [default] org.codehaus.groovy#groovy-all;2.4.4 from central in [default] org.codehaus.jackson#jackson-core-asl;1.9.13 from central in [default] org.codehaus.jackson#jackson-jaxrs;1.9.13 from central in [default] org.codehaus.jackson#jackson-mapper-asl;1.9.13 from central in [default] org.codehaus.jackson#jackson-xc;1.9.13 from central in [default] org.codehaus.janino#commons-compiler;2.7.6 from central in [default] org.codehaus.janino#janino;2.7.6 from central in [default] org.codehaus.jettison#jettison;1.1 from central in [default] org.datanucleus#datanucleus-api-jdo;4.2.4 from central in [default] org.datanucleus#datanucleus-core;4.1.17 from central in [default] org.datanucleus#datanucleus-rdbms;4.1.19 from central in [default] org.datanucleus#javax.jdo;3.2.0-m3 from central in [default] org.eclipse.jetty.aggregate#jetty-all;7.6.0.v20120127 from central in [default] org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016 from central in [default] org.fusesource.leveldbjni#leveldbjni-all;1.8 from central in [default] org.hamcrest#hamcrest-core;1.3 from central in [default] org.jruby.jcodings#jcodings;1.0.8 from central in [default] org.jruby.joni#joni;2.1.2 from central in [default] org.mortbay.jetty#jetty;6.1.26 from central in [default] org.mortbay.jetty#jetty-util;6.1.26 from central in [default] org.openjdk.jol#jol-core;0.2 from central in [default] org.slf4j#slf4j-api;1.7.36 from central in [default] org.slf4j#slf4j-log4j12;1.7.14 from central in [default] org.sonatype.sisu.inject#cglib;2.2.1-v20090111 from central in [default] org.tukaani#xz;1.5 from central in [default] org.xerial.snappy#snappy-java;1.1.8.2 from central in [default] stax#stax-api;1.0.1 from central in [default] tomcat#jasper-compiler;5.5.23 from central in [default] tomcat#jasper-runtime;5.5.23 from central in [default] xmlenc#xmlenc;0.52 from central in [default] :: evicted modules: org.slf4j#slf4j-api;1.7.10 by [org.slf4j#slf4j-api;1.7.36] in [default] org.slf4j#slf4j-log4j12;1.6.1 by [org.slf4j#slf4j-log4j12;1.7.14] in [default] log4j#log4j;1.2.16 by [log4j#log4j;1.2.17] in [default] commons-logging#commons-logging;1.1.3 by [commons-logging#commons-logging;1.2] in [default] asm#asm;3.1 by [asm#asm;3.2] in [default] io.dropwizard.metrics#metrics-core;3.1.2 by [io.dropwizard.metrics#metrics-core;3.1.0] in [default] org.xerial.snappy#snappy-java;1.1.1.3 by [org.xerial.snappy#snappy-java;1.1.8.2] in [default] com.google.code.findbugs#jsr305;3.0.0 by [com.google.code.findbugs#jsr305;3.0.2] in [default] javax.servlet#servlet-api;2.4 by [javax.servlet#servlet-api;2.5] in [default] commons-logging#commons-logging;1.0.3 by [commons-logging#commons-logging;1.2] in [default] org.apache.hadoop#hadoop-auth;2.5.1 by [org.apache.hadoop#hadoop-auth;2.7.2] in [default] io.netty#netty;3.6.2.Final by [io.netty#netty;3.7.0.Final] in [default] com.google.code.findbugs#jsr305;2.0.1 by [com.google.code.findbugs#jsr305;3.0.0] in [default] --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 176 | 168 | 168 | 13 || 164 | 164 | --------------------------------------------------------------------- :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS :: retrieving :: org.apache.spark#spark-submit-parent-d0d2962d-ae27-4526-a0c7-040a542e1e54 confs: [default] 164 artifacts copied, 0 already retrieved (195815kB/314ms) +--------------------+ | namespace| +--------------------+ | a1| | a1_dev| | a2| | access| | access_xzx01| | access_xzx02| | activity| | ...| +--------------------+ only showing top 20 rows scala> ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #42599 from pan3793/unshare-guava. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

dongjoon-hyun · 2024-05-10T14:59:31Z

Thank you for updating this, @pan3793 .

dongjoon-hyun · 2024-05-10T15:01:22Z

...lyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/IntervalExpressionsSuite.scala

@@ -341,7 +341,7 @@ class IntervalExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {

    Seq(
      (Period.ofMonths(2), Int.MaxValue) -> "overflow",
-      (Period.ofMonths(Int.MinValue), 10d) -> "not in range",
+      (Period.ofMonths(Int.MinValue), 10d) -> "out of range",


So, this error message is originated from Guava?

cc @cloud-fan do you think such error message change is acceptable?

LuciferYang · 2024-05-16T03:23:58Z

should we test 33.2.0?

33.1.0-jre is chosen because Spark uses 33.1.0-jre in other places, and after this change, we can unify all Guava versions.

OK

LuciferYang · 2024-05-16T06:13:44Z

Is there a possibility that upgrading Guava could carry similar risks as described in #46528 ? Could there be legacy Hive UDF jars that are bound to Guava 14.0.1?

WDYT @dongjoon-hyun ?

LuciferYang · 2024-05-23T02:50:54Z

Should we continue to push this one forward？

LuciferYang · 2024-07-12T09:04:50Z

dev/deps/spark-deps-hadoop-3-hive-2.3

 flatbuffers-java/23.5.26//flatbuffers-java-23.5.26.jar
 gcs-connector/hadoop3-2.2.21/shaded/gcs-connector-hadoop3-2.2.21-shaded.jar
 gmetric4j/1.0.10//gmetric4j-1.0.10.jar
 gson/2.2.4//gson-2.2.4.jar
-guava/14.0.1//guava-14.0.1.jar
+guava/33.1.0-jre//guava-33.1.0-jre.jar


we can upgrade to test 33.2.1 now

dongjoon-hyun

Could you rebase this PR once more, @pan3793 ?

I'm wondering if we want to deliver this via Apache Spark 4.0.0-preview2 next Monday.

pan3793 · 2024-09-11T18:27:57Z

@dongjoon-hyun I rebased this PR and updated the description. Guava upgrading was requested many times from various users, it would be great if we could deliver this Guava upgradig in the coming Spark 4.0.0-preview2

dongjoon-hyun · 2024-09-11T18:29:10Z

I agree with you, @pan3793 . I hope we can try in preview2.

dongjoon-hyun

~~BTW, the latest one is 33.3.0-jre. Could you update this PR if there is no reason to stick to old one~~, @pan3793 ?

https://mvnrepository.com/artifact/com.google.guava/guava/33.3.0-jre

Never mind. I saw that it's consistent with the existing Guava version.

spark/pom.xml

Line 294 in e037953

<connect.guava.version>33.2.1-jre</connect.guava.version>

dongjoon-hyun · 2024-09-12T16:24:22Z

dev/deps/spark-deps-hadoop-3-hive-2.3

@@ -33,6 +33,7 @@ breeze-macros_2.13/2.1.0//breeze-macros_2.13-2.1.0.jar
 breeze_2.13/2.1.0//breeze_2.13-2.1.0.jar
 bundle/2.24.6//bundle-2.24.6.jar
 cats-kernel_2.13/2.8.0//cats-kernel_2.13-2.8.0.jar
+checker-qual/3.42.0//checker-qual-3.42.0.jar


I guess we need to update LICENSE-binary and NOTICE-binary for new jars, @pan3793 .

Please proceed as a follow-up, @pan3793 .

dongjoon-hyun

+1, LGTM.

dongjoon-hyun · 2024-09-12T16:27:29Z

Merged to master for Apache Spark 4.0.0-preview2.

dongjoon-hyun · 2024-09-12T16:31:17Z

Thank you, @pan3793 and @LuciferYang .

I want to ping you guys for head-ups once more, @JoshRosen , @cloud-fan , @HyukjinKwon , @yaooqinn , @mridulm , @viirya , @huaxingao .

### What changes were proposed in this pull request? This PR upgrades Spark's built-in Guava from 14 to 33.2.1-jre Currently, Spark uses Guava 14 because the previous built-in Hive 2.3.9 is incompatible with new Guava versions. HIVE-27560 (apache/hive#4542) makes Hive 2.3.10 compatible with Guava 14+ (thanks to LuciferYang) ### Why are the changes needed? It's a long-standing issue, see prior discussions at apache#35584, apache#36231, and apache#33989 ### Does this PR introduce _any_ user-facing change? Yes, some user-faced error messages changed. ### How was this patch tested? GA passed. Closes apache#42493 from pan3793/guava. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

This PR upgrades Spark's built-in Guava from 14 to 33.2.1-jre Currently, Spark uses Guava 14 because the previous built-in Hive 2.3.9 is incompatible with new Guava versions. HIVE-27560 (apache/hive#4542) makes Hive 2.3.10 compatible with Guava 14+ (thanks to LuciferYang) It's a long-standing issue, see prior discussions at apache#35584, apache#36231, and apache#33989 Yes, some user-faced error messages changed. GA passed. Closes apache#42493 from pan3793/guava. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 1f24b2d)

This PR upgrades Spark's built-in Guava from 14 to 33.2.1-jre Currently, Spark uses Guava 14 because the previous built-in Hive 2.3.9 is incompatible with new Guava versions. HIVE-27560 (apache/hive#4542) makes Hive 2.3.10 compatible with Guava 14+ (thanks to LuciferYang) It's a long-standing issue, see prior discussions at apache#35584, apache#36231, and apache#33989 Yes, some user-faced error messages changed. GA passed. Closes apache#42493 from pan3793/guava. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 1f24b2d) (cherry picked from commit e5cc252)

This PR upgrades Spark's built-in Guava from 14 to 33.2.1-jre Currently, Spark uses Guava 14 because the previous built-in Hive 2.3.9 is incompatible with new Guava versions. HIVE-27560 (apache/hive#4542) makes Hive 2.3.10 compatible with Guava 14+ (thanks to LuciferYang) It's a long-standing issue, see prior discussions at apache#35584, apache#36231, and apache#33989 Yes, some user-faced error messages changed. GA passed. Closes apache#42493 from pan3793/guava. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 1f24b2d)

github-actions bot added SQL BUILD labels Aug 15, 2023

pan3793 force-pushed the guava branch from b12683d to 7460d1d Compare August 15, 2023 02:09

pan3793 commented Aug 15, 2023

View reviewed changes

pom.xml Outdated Show resolved Hide resolved

pan3793 force-pushed the guava branch from 7460d1d to cabbcc6 Compare August 15, 2023 05:25

pan3793 mentioned this pull request Aug 16, 2023

HIVE-27560: [2.3] Enhancing compatibility with Guava apache/hive#4542

Merged

pan3793 mentioned this pull request Aug 22, 2023

[SPARK-45292][SQL][HIVE] Remove Guava from shared classes from IsolatedClientLoader #42599

Closed

github-actions bot added the Stale label Dec 1, 2023

github-actions bot closed this Dec 2, 2023

ulysses-you reopened this May 10, 2024

ulysses-you removed the Stale label May 10, 2024

pan3793 force-pushed the guava branch from cabbcc6 to 502ade2 Compare May 10, 2024 11:03

github-actions bot added the CORE label May 10, 2024

dongjoon-hyun reviewed May 10, 2024

View reviewed changes

pan3793 changed the title ~~[SPARK-44811][BUILD] Upgrade Guava to 32+~~ [SPARK-44811][BUILD] Upgrade Guava to 33.1.0-jre May 10, 2024

pan3793 mentioned this pull request Jul 2, 2024

[SPARK-48763][CONNECT][BUILD] Move connect server and common to builtin module #47157

Closed

pan3793 force-pushed the guava branch from 58e909c to 7395668 Compare July 12, 2024 04:34

LuciferYang reviewed Jul 12, 2024

View reviewed changes

pan3793 force-pushed the guava branch from 7395668 to cdef975 Compare July 15, 2024 03:37

dongjoon-hyun reviewed Sep 11, 2024

View reviewed changes

pan3793 added 2 commits September 12, 2024 02:19

[SPARK-44811][BUILD] Bump Guava 33.2.1-jre

e14900d

nit

c75e76a

pan3793 force-pushed the guava branch from cdef975 to c75e76a Compare September 11, 2024 18:21

pan3793 changed the title ~~[SPARK-44811][BUILD] Upgrade Guava to 33.1.0-jre~~ [SPARK-44811][BUILD] Upgrade Guava to 33.2.1-jre Sep 11, 2024

dongjoon-hyun reviewed Sep 11, 2024

View reviewed changes

dongjoon-hyun reviewed Sep 12, 2024

View reviewed changes

dongjoon-hyun approved these changes Sep 12, 2024

View reviewed changes

dongjoon-hyun closed this in 1f24b2d Sep 12, 2024

panbingkun mentioned this pull request Sep 25, 2024

[SPARK-49774][BUILD] Upgrade guava to 33.3.1-jre #48233

Closed

[SPARK-44811][BUILD] Upgrade Guava to 33.2.1-jre #42493

[SPARK-44811][BUILD] Upgrade Guava to 33.2.1-jre #42493

Uh oh!

Conversation

pan3793 commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

Uh oh!

LuciferYang commented Aug 15, 2023

Uh oh!

pan3793 commented Aug 15, 2023

Uh oh!

srowen commented Aug 15, 2023

Uh oh!

pan3793 commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sunchao commented Aug 16, 2023

Uh oh!

pan3793 commented Aug 16, 2023

Uh oh!

pan3793 commented Aug 16, 2023

Uh oh!

xkrogen commented Aug 16, 2023

Uh oh!

cxzl25 commented Aug 21, 2023

Uh oh!

JoshRosen commented Aug 21, 2023

Uh oh!

pan3793 commented Aug 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 1, 2023

Uh oh!

pan3793 commented Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dongjoon-hyun commented May 10, 2024

Uh oh!

dongjoon-hyun May 10, 2024

Choose a reason for hiding this comment

Uh oh!

pan3793 May 10, 2024

Choose a reason for hiding this comment

Uh oh!

pan3793 May 15, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang commented May 16, 2024

Uh oh!

LuciferYang commented May 16, 2024

Uh oh!

LuciferYang commented May 23, 2024

Uh oh!

LuciferYang Jul 12, 2024

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

pan3793 commented Sep 11, 2024

Uh oh!

dongjoon-hyun commented Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dongjoon-hyun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Sep 12, 2024

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Sep 12, 2024

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment

pan3793 commented Aug 15, 2023 •

edited

Loading

pan3793 commented Aug 15, 2023 •

edited

Loading

pan3793 commented Aug 22, 2023 •

edited

Loading

pan3793 commented Dec 18, 2023 •

edited

Loading

dongjoon-hyun commented Sep 11, 2024 •

edited

Loading

dongjoon-hyun left a comment •

edited

Loading