[HOT FIX] MiMa tests are broken

[andrewor14]

This is blocking #3353 and other patches.

[SparkQA]

Test build #23623 has started for PR 3371 at commit c4d4f4e.

• This patch merges cleanly.

[pwendell]

LGTM - we should add a fix for MIMA where we can mark interfaces that we don't want users to extend, and it will ignore them.

[SparkQA]

Test build #23623 has finished for PR 3371 at commit c4d4f4e.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class LinearBinaryClassificationModel(LinearModel):
  • class LogisticRegressionModel(LinearBinaryClassificationModel):
  • class LogisticRegressionWithLBFGS(object):
  • class SVMModel(LinearBinaryClassificationModel):
  • class Rating(namedtuple("Rating", ["user", "product", "rating"])):
  • class RDDRangeSampler(RDDSamplerBase):
  • class SizeLimitedStream(object):
  • class CompressedStream(object):
  • class LargeObjectSerializer(Serializer):
  • class CompressedSerializer(Serializer):

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23623/
Test FAILed.

[vanzin]

Not a comment on the change itself, but sometimes these mima errors confuse me. I have a couple of questions:

[error] * abstract method submissionTime()Long in interface org.apache.spark.SparkStageInfo does not have a correspondent in old version

Why is adding a new method an error? That is fine as far as API compatibility is concerned, right? Or is this restricted to interfaces (because of risk of AbstractMethodError)?

[error] * method this(Int,Int,java.lang.String,Int,Int,Int,Int)Unit in class org.apache.spark.SparkStageInfoImpl does not have a correspondent in new version

SparkStageInfoImpl is a private class, why isn't it being ignored?

[vanzin]

master is now v1.3 so these should probably be moved to the right "case" above.

[andrewor14]

ahh... that's why it's failing.

[pwendell]

Hey @vanzin. For an interface, adding a new method does break existing implementations. This is an interface we never expect users to extend, so we need a way in general to annotate these and have mima ignore them. Something like @ReadOnlyInterface or something would be nice for this.

For the second issue, I have no idea what is going on with that.

[SparkQA]

Test build #23631 has started for PR 3371 at commit 842d059.

• This patch merges cleanly.

[pwendell]

I tested this locally and it worked, I'm going to pull this in since the current state of affairs is a huge disruption.

[SparkQA]

Test build #23631 has finished for PR 3371 at commit 842d059.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23631/
Test PASSed.