MLeap compatibility with latest version of Scala (currently 2.12.x)?

[cappaberra]

Scala version 2.11.x (the latest mentioned in your README -- see quote below) is no longer being maintained. To ensure production viability in the future, I wanted to find out if MLeap will work with Scala v2.12.x?

MLeap is cross-compiled for Scala 2.10 and 2.11. Because we depend heavily on Typesafe config for MLeap, we only support Java 8 at the moment. This means the following configurations should be possible:

• Scala 2.10 and Java 8
• Scala 2.11 and Java 8

This additional information in the Scala release notes is good context as well:

From 2.11.1:

The Scala 2.11.x series targets Java 6, with (evolving) experimental support for Java 8. In 2.11.1, Java 8 support is mostly limited to reading Java 8 bytecode and parsing Java 8 source. Stay tuned for more complete (experimental) Java 8 support. The next major release, 2.12, will most likely target Java 8 by default.

From 2.11.12:

We strongly encourage you to upgrade to the latest stable version of Scala 2.12.x, as the 2.11.x series is no longer actively maintained.

So, would the combination "Scala 2.12 and Java 8" work with MLeap?

Thanks!

[cappaberra]

I compiled MLeap's Runtime with Scala 2.12.6 (latest as of this writing) without any errors and ran it with the example (AirBnB) linear regression model / data without any known issues. So, looks like it's ok with Scala 2.12.x from what I can see, but would love your expert opinions! Thanks in advance!

# echo 'enablePlugins(UniversalPlugin, LinuxPlugin, DebianPlugin)

name := "mleap"
version := "0.9.6"
maintainer := "Hello World <hello@world.com>"
packageSummary := "MLeap Runtime Microservice"
packageDescription := "MLeap Runtime Microservice"' >> mleap-serving/build.sbt

# Compile Scala code and build Debian Package
# sbt mleap-serving/debian:packageBin
[info] Loading project definition from /var/tmp/mleap/project
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[info] Set current project to mleap (in build file:/var/tmp/mleap/)
[info] Wrote /var/tmp/mleap/mleap-serving/target/scala-2.11/mleap_2.11-0.9.6.pom
[info] Compiling 3 protobuf files to /var/tmp/mleap/bundle-ml/target/scala-2.11/src_managed/main
[info] Compiling schema /var/tmp/mleap/bundle-protobuf/bundle.proto
[info] Compiling schema /var/tmp/mleap/bundle-protobuf/ext/dtree.proto
[info] Compiling schema /var/tmp/mleap/bundle-protobuf/ext/ctree.proto
protoc-jar: protoc version: 320, detected platform: linux/amd64
protoc-jar: executing: [/tmp/protocjar8665130219312021696/bin/protoc.exe, --plugin=protoc-gen-scala=/tmp/protocbridge1730641838538421320, --scala_out=flat_package,grpc:/var/tmp/mleap/bundle-ml/target/scala-2.11/src_managed/main, -I/var/tmp/mleap/bundle-protobuf, /var/tmp/mleap/bundle-protobuf/bundle.proto, /var/tmp/mleap/bundle-protobuf/ext/dtree.proto, /var/tmp/mleap/bundle-protobuf/ext/ctree.proto]
[info] Compiling protobuf
[info] Protoc target directory: /var/tmp/mleap/bundle-ml/target/scala-2.11/src_managed/main
[info] Compiling 51 Scala sources to /var/tmp/mleap/bundle-ml/target/scala-2.11/classes...
[info] Compiling 156 Scala sources and 3 Java sources to /var/tmp/mleap/mleap-runtime/target/scala-2.11/classes...
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/Pipeline.scala:5: imported `Transformer' is permanently hidden by definition of type Transformer in package transformer
[warn] import ml.combust.mleap.runtime.frame.{FrameBuilder, Transformer}
[warn]                                                      ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/MleapSupport.scala:26: type DefaultLeapFrame in package runtime is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.DefaultLeapFrame instead
[warn]     def toLeapFrame: DefaultLeapFrame = LeapFrameConverter.convert(data)
[warn]                      ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/MleapSupport.scala:30: type DefaultLeapFrame in package runtime is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.DefaultLeapFrame instead
[warn]     def toLeapFrame: DefaultLeapFrame = LeapFrameConverter.convert(data)
[warn]                      ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/MleapSupport.scala:33: type LeapFrame in package runtime is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.LeapFrame instead
[warn]   implicit class MleapLeapFrameOps[LF <: LeapFrame[LF]](frame: LF) {
[warn]                                          ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/Pipeline.scala:13: type Transformer in package transformer is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.Transformer instead
[warn] case class PipelineModel(transformers: Seq[Transformer]) extends Model {
[warn]                                        ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/Pipeline.scala:13: type Transformer in package transformer is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.Transformer instead
[warn] case class PipelineModel(transformers: Seq[Transformer]) extends Model {
[warn]            ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/Pipeline.scala:24: type Transformer in package transformer is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.Transformer instead
[warn]                     override val model: PipelineModel) extends Transformer {
[warn]                                                                ^
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/Pipeline.scala:25: type Transformer in package transformer is deprecated: this type will be removed for MLeap 1.0, use ml.combust.mleap.runtime.frame.Transformer instead
[warn]   def transformers: Seq[Transformer] = model.transformers
[warn]                     ^
[warn] 8 warnings found
[info] Compiling 7 Scala sources and 10 Java sources to /var/tmp/mleap/mleap-xgboost-java/target/scala-2.11/classes...
[info] Compiling 6 Scala sources to /var/tmp/mleap/mleap-avro/target/scala-2.11/classes...
[info] Compiling 10 Scala sources to /var/tmp/mleap/mleap-serving/target/scala-2.11/classes...
[info] Main Scala API documentation to /var/tmp/mleap/mleap-serving/target/scala-2.11/api...
model contains 29 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-serving/target/scala-2.11/mleap_2.11-0.9.6-javadoc.jar ...
[info] Packaging /var/tmp/mleap/mleap-serving/target/scala-2.11/mleap_2.11-0.9.6.jar ...
[info] Done packaging.
[info] Done packaging.
[info] Main Scala API documentation to /var/tmp/mleap/mleap-base/target/scala-2.11/api...
[info] Wrote /var/tmp/mleap/mleap-base/target/scala-2.11/mleap-base_2.11-0.9.6.pom
[info] Packaging /var/tmp/mleap/mleap-base/target/scala-2.11/mleap-base_2.11-0.9.6-sources.jar ...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/mleap-base/target/scala-2.11/mleap-base_2.11-0.9.6.jar ...
[info] Done packaging.
model contains 7 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-base/target/scala-2.11/mleap-base_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/mleap-tensor/target/scala-2.11/mleap-tensor_2.11-0.9.6-sources.jar ...
[info] Wrote /var/tmp/mleap/mleap-tensor/target/scala-2.11/mleap-tensor_2.11-0.9.6.pom
[info] Done packaging.
[info] Main Scala API documentation to /var/tmp/mleap/mleap-tensor/target/scala-2.11/api...
[info] Packaging /var/tmp/mleap/mleap-tensor/target/scala-2.11/mleap-tensor_2.11-0.9.6.jar ...
[info] Done packaging.
model contains 12 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-tensor/target/scala-2.11/mleap-tensor_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[warn] Skipped generating '<exclusion/>' for org.scalatest#*. Dependency exclusion should have both 'org' and 'module' to comply with Maven POM's schema.
[info] Wrote /var/tmp/mleap/mleap-core/target/scala-2.11/mleap-core_2.11-0.9.6.pom
[info] Packaging /var/tmp/mleap/mleap-core/target/scala-2.11/mleap-core_2.11-0.9.6-sources.jar ...
[info] Main Scala API documentation to /var/tmp/mleap/mleap-core/target/scala-2.11/api...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/mleap-core/target/scala-2.11/mleap-core_2.11-0.9.6.jar ...
[info] Done packaging.
model contains 243 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-core/target/scala-2.11/mleap-core_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[info] Wrote /var/tmp/mleap/bundle-ml/target/scala-2.11/bundle-ml_2.11-0.9.6.pom
[info] Main Scala API documentation to /var/tmp/mleap/bundle-ml/target/scala-2.11/api...
[info] Packaging /var/tmp/mleap/bundle-ml/target/scala-2.11/bundle-ml_2.11-0.9.6-sources.jar ...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/bundle-ml/target/scala-2.11/bundle-ml_2.11-0.9.6.jar ...
[info] Done packaging.
model contains 210 documentable templates
[warn] /var/tmp/mleap/bundle-ml/src/main/scala/ml/combust/bundle/dsl/Value.scala:24: Could not find any member to link for "ml.combust.mleap.tensor.Tensor".
[warn]   /** Convert [[ml.combust.mleap.tensor.Tensor]] to [[ml.bundle.Tensor]].
[warn]   ^
[warn] /var/tmp/mleap/bundle-ml/src/main/scala/ml/combust/bundle/dsl/Attributes.scala:72: Could not find any member to link for "(String,".
[warn]   /** Construct an [[Attributes]] from a list of [[(String, Value)]].
[warn]   ^
[warn] /var/tmp/mleap/bundle-ml/src/main/scala/ml/combust/bundle/dsl/Attributes.scala:63: Could not find any member to link for "(String,".
[warn]   /** Construct an [[Attributes]] from a list of [[(String, Value)]].
[warn]   ^
[warn] three warnings found
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/bundle-ml/target/scala-2.11/bundle-ml_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[info] Wrote /var/tmp/mleap/mleap-runtime/target/scala-2.11/mleap-runtime_2.11-0.9.6.pom
[info] Packaging /var/tmp/mleap/mleap-runtime/target/scala-2.11/mleap-runtime_2.11-0.9.6-sources.jar ...
[info] Main Scala API documentation to /var/tmp/mleap/mleap-runtime/target/scala-2.11/api...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/mleap-runtime/target/scala-2.11/mleap-runtime_2.11-0.9.6.jar ...
[info] Done packaging.
[warn] /var/tmp/mleap/mleap-runtime/src/main/scala/ml/combust/mleap/runtime/transformer/Pipeline.scala:5: imported `Transformer' is permanently hidden by definition of type Transformer in package transformer
[warn] import ml.combust.mleap.runtime.frame.{FrameBuilder, Transformer}
[warn]                                                      ^
model contains 229 documentable templates
[warn] one warning found
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-runtime/target/scala-2.11/mleap-runtime_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[info] Main Scala API documentation to /var/tmp/mleap/mleap-avro/target/scala-2.11/api...
[info] Wrote /var/tmp/mleap/mleap-avro/target/scala-2.11/mleap-avro_2.11-0.9.6.pom
[info] Packaging /var/tmp/mleap/mleap-avro/target/scala-2.11/mleap-avro_2.11-0.9.6.jar ...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/mleap-avro/target/scala-2.11/mleap-avro_2.11-0.9.6-sources.jar ...
[info] Done packaging.
model contains 11 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-avro/target/scala-2.11/mleap-avro_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[info] Wrote /var/tmp/mleap/mleap-xgboost-java/target/scala-2.11/mleap-xgboost-java_2.11-0.9.6.pom
[info] Packaging /var/tmp/mleap/mleap-xgboost-java/target/scala-2.11/mleap-xgboost-java_2.11-0.9.6-sources.jar ...
[info] Done packaging.
[info] Packaging /var/tmp/mleap/mleap-xgboost-java/target/scala-2.11/mleap-xgboost-java_2.11-0.9.6.jar ...
[info] Main Scala API documentation to /var/tmp/mleap/mleap-xgboost-java/target/scala-2.11/api...
[info] Done packaging.
model contains 36 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /var/tmp/mleap/mleap-xgboost-java/target/scala-2.11/mleap-xgboost-java_2.11-0.9.6-javadoc.jar ...
[info] Done packaging.
[info] Building debian package with native implementation
[info] dpkg-deb: building package 'mleap' in '../mleap_0.9.6_all.deb'.
[success] Total time: 148 s, completed May 3, 2018 1:33:01 PM

[hollinwilkins]

@cappaberra The MLeap runtime should be fine with 2.12, and in fact I would love to have this on Maven Central. The path to getting there is to upgrade to the latest SBT, which supports crossCompile settings on a per submodule basis (anything that depends on Spark cannot compile for 2.12). The only other concern I would have is our dependency on spark-mllib-local, which is not compiled for 2.12. So we would need to get rid of that dependency as well.

[danielyli]

Any updates on this?

[mailming]

Half year passed, is there any other solution or progress on this?

[marsbible]

Any updates on this? Scala 2.11 is no longer actively maintained.

[ancasarb]

Closing this, there's another ticket for scala 2.12 support as well (#445). PR to add support for scala 2.12 should be merged soon #496.