update the version of the dependencies not available on the central maven repository

[dportabella]

Thanks for publishing a release.

however, the release still depends on dependencies not available on the central maven repository:

$ spark-shell --master local[4] --packages "io.archivesunleashed:aut:0.10.0"
Exception in thread "main" java.lang.RuntimeException: [
unresolved dependency: com.sleepycat#je;4.1.6: not found,
unresolved dependency: org.apache.hadoop#hadoop-core;0.20.2-cdh3u4: not found
]

you might need to remove your repository cache directory to see the problem (~/.m2 and ~/.ivy2). make sure to make a backup just in case.

could you please update the version of your dependencies, specially these two which are probably causing problems, and make a new release?

$ mvn dependency:tree -Dverbose
[INFO] +- org.netpreserve.openwayback:openwayback-core:jar:2.0.0.BETA.2:compile
[INFO] |  \- com.sleepycat:je:jar:4.1.6:compile
[INFO] +- org.netpreserve.commons:webarchive-commons:jar:1.1.4:compile
[INFO] |  +- org.apache.hadoop:hadoop-core:jar:0.20.2-cdh3u4:compile

[dportabella]

Hi, thx for looking at this.
I am not sure I am testing it correctly.
I've tested it as follows, but it still fails:

docker run -it -p 8088:8088 -p 8042:8042 -h sandbox sequenceiq/spark:1.6.0 bash

# git
sudo yum install -y git

# wget
yum install -y wget

# jdk8: https://tecadmin.net/install-java-8-on-centos-rhel-and-fedora/
cd /opt/
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u151-b12/e758a0de34e24606bca991d704f6dcbf/jdk-8u151-linux-x64.tar.gz"
tar xzf jdk-8u151-linux-x64.tar.gz
cd /opt/jdk1.8.0_151/
alternatives --install /usr/bin/java java /opt/jdk1.8.0_151/bin/java 2
alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_151/bin/jar 2
alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_151/bin/javac 2
alternatives --config java
alternatives --config jar
alternatives --config javac
export JAVA_HOME=/opt/jdk1.8.0_151
export JRE_HOME=/opt/jdk1.8.0_151/jre
export PATH=$PATH:/opt/jdk1.8.0_151/bin:/opt/jdk1.8.0_151/jre/bin

# maven
cd ~
curl http://www-eu.apache.org/dist/maven/maven-3/3.5.2/binaries/apache-maven-3.5.2-bin.tar.gz | tar -xvzf -

git clone https://github.com/archivesunleashed/aut.git
cd aut
git checkout issue-111
../apache-maven-3.5.2/bin/mvn install -DskipTests

spark-shell --master local[4] --packages "io.archivesunleashed:aut:0.10.1-SNAPSHOT"
	found io.archivesunleashed#aut;0.10.1-SNAPSHOT in local-m2-cache
:: problems summary ::
:::: WARNINGS
		[NOT FOUND  ] commons-net#commons-net;1.4.1!commons-net.jar (2ms)
		[NOT FOUND  ] org.codehaus.jackson#jackson-core-asl;1.5.2!jackson-core-asl.jar (1ms)
		[NOT FOUND  ] org.codehaus.jackson#jackson-mapper-asl;1.5.2!jackson-mapper-asl.jar (0ms)
		[NOT FOUND  ] net.java.dev.jets3t#jets3t;0.6.1!jets3t.jar (0ms)

[ruebot]

@dportabella if I'm understanding things correctly, your use case is that you'd like to be able to load aut with --packages instead of --jars with spark-shell?

[dportabella]

testing spark-shell with --packages instead of --jars is a way to verify that the auth library can be imported in a sbt project.
I have a sbt project that uses scala and spark, with some other dependencies. I need to add the auth dependency inside the build.sbt. I could include your external fat jar in my project, but this is a poor practise. Moreover, these old dependencies have many issues and conflicts with other dependencies.

[ruebot]

@dportabella ok, so we're resolved the original issue you created then, "update the version of the dependencies not available on the central maven repository", right? If so, can you create a separate issue with your use case and I'll see what I can do there. But, my hands might be tied with upstream issues like #23.

[dportabella]

your pom.xml still depends on this archive.org repository, meaning the dependencies do not exist in the maven central anymore (maybe because does library versions had important security issues).
the idea of this ticket is to update to versions of the dependencies, so that they are found in the maven central repository.

    <repository>
      <id>internetarchive</id>
      <name>Internet Archive Maven Repository</name>
      <url>http://builds.archive.org:8080/maven2</url>
    </repository>

[ruebot]

@dportabella those repositories are removed in the PR associated with this issue. https://github.com/archivesunleashed/aut/pull/112/files#diff-600376dffeb79835ede4a0b285078036L66. Was your testing on the PR branch?

[dportabella]

ah, yes, sorry.
yes, yes, i tested on the issue-111 branch.
then, i don't know why the spark-shell task still fails. i'll open a new issue.