Skip to content

[SPARK-16770][BUILD] Fix JLine dependency management and version (Sca… #14429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

stsc-pentasys
Copy link
Contributor

What changes were proposed in this pull request?

As of Scala 2.11.x there is no longer a org.scala-lang:jline version aligned to the scala version itself. Scala console now uses the plain jline:jline module. Spark's dependency management did not reflect this change properly, causing Maven to pull in Jline via transitive dependency. Unfortunately Jline 2.12 contained a minor but very annoying bug rendering the shell almost useless for developers with german keyboard layout. This request contains the following chages:

  • Exclude transitive dependency 'jline:jline' from hive-exec module
  • Remove global properties 'jline.version' and 'jline.groupId'
  • Add both properties and dependency to 'scala-2.11' profile
  • Add explicit dependency on 'jline:jline' to module 'spark-repl'

How was this patch tested?

  • Running mvn dependency:tree and checking for correct Jline version 2.12.1
  • Running full builds with assembly and checking for jline-2.12.1.jar in 'lib' folder of generated tarball

</properties>
<dependencyManagement>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've added a new dependencyManagement section which won't work. Add to the existing one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just aligned it with the profile for scala 2.10.x, And from my point of view it works. Includes the right Jline version in the delivered tarball.

@srowen
Copy link
Member

srowen commented Jul 31, 2016

This will fail to build because the deps/* files haven't been updated. The dev/test-dependencies.sh script can do this.

@srowen
Copy link
Member

srowen commented Aug 3, 2016

OK @stsc-pentasys I think I'm on board with this after I reminded myself of how it works. One last suggestion. It seems like this should be in dependencyManagment outside of any profile:

          <dependency>
            <groupId>${jline.groupid}</groupId>
            <artifactId>jline</artifactId>
            <version>${jline.version}</version>
          </dependency>

That much is constant. It's the version and group that vary with Scala, and the profiles already set those.

@stsc-pentasys
Copy link
Contributor Author

That's of course the cleanest solution.

@srowen
Copy link
Member

srowen commented Aug 3, 2016

Yep that LGTM

@srowen
Copy link
Member

srowen commented Aug 3, 2016

Jenkins test this please

@SparkQA
Copy link

SparkQA commented Aug 3, 2016

Test build #63184 has finished for PR 14429 at commit 35cc6c3.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 3, 2016

Test build #63185 has finished for PR 14429 at commit 35cc6c3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen
Copy link
Member

srowen commented Aug 4, 2016

Merged to master/2.0

asfgit pushed a commit that referenced this pull request Aug 4, 2016
## What changes were proposed in this pull request?
As of Scala 2.11.x there is no longer a org.scala-lang:jline version aligned to the scala version itself. Scala console now uses the plain jline:jline module. Spark's  dependency management did not reflect this change properly, causing Maven to pull in Jline via transitive dependency. Unfortunately Jline 2.12 contained a minor but very annoying bug rendering the shell almost useless for developers with german keyboard layout. This request contains the following chages:
- Exclude transitive dependency 'jline:jline' from hive-exec module
- Remove global properties 'jline.version' and 'jline.groupId'
- Add both properties and dependency to 'scala-2.11' profile
- Add explicit dependency on 'jline:jline' to  module 'spark-repl'

## How was this patch tested?
- Running mvn dependency:tree and checking for correct Jline version 2.12.1
- Running full builds with assembly and checking for jline-2.12.1.jar in 'lib' folder of generated tarball

Author: Stefan Schulze <stefan.schulze@pentasys.de>

Closes #14429 from stsc-pentasys/SPARK-16770.

(cherry picked from commit 4775eb4)
Signed-off-by: Sean Owen <sowen@cloudera.com>
@asfgit asfgit closed this in 4775eb4 Aug 4, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants