[SPARK-13986][CORE][MLLIB] Remove DeveloperApi-annotations for non-publics

[dongjoon-hyun]

What changes were proposed in this pull request?

Spark uses @DeveloperApi annotation, but sometimes it seems to conflict with visibility. This PR tries to fix those conflict by removing annotations for non-publics. The following is the example.

JobResult.scala

@DeveloperApi
sealed trait JobResult

@DeveloperApi
case object JobSucceeded extends JobResult

-@DeveloperApi
private[spark] case class JobFailed(exception: Exception) extends JobResult

How was this patch tested?

Pass the existing Jenkins test.

[SparkQA]

Test build #53474 has finished for PR 11797 at commit 6f3ea7a.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class JobFailed(exception: Exception) extends JobResult
  • trait ValidatorParams extends Params
  • class MatrixUDT extends UserDefinedType[Matrix]
  • case class NodeIndexUpdater(
  • class NodeIdCache(
  • class ImpurityStats(

[dongjoon-hyun]

Just rebased.

[jodersky]

I agree there are discrepancies between the intended meaning of annotations and access modifiers, however my reaction would be to keep everything private and make stuff public on an actual requirement basis.

Making things public is easy, however reverting things to private once they're in the wild is hard and can have many unforeseen consequences. So my recommendation is to remove the @DeveloperAPI annotation but keep the current access scope.

[rxin]

+1 to what @jodersky said.

[SparkQA]

Test build #53486 has finished for PR 11797 at commit ec7fff2.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class JobFailed(exception: Exception) extends JobResult
  • trait ValidatorParams extends Params
  • class MatrixUDT extends UserDefinedType[Matrix]
  • case class NodeIndexUpdater(
  • class NodeIdCache(
  • class ImpurityStats(

[dongjoon-hyun]

Thank you, @jodersky and @rxin . I'll do like that.

Just one more question: JobFailed is needed to be private[spark]?

[dongjoon-hyun]

I updated this PR and JIRA according to the comments. Thank you all.

[SparkQA]

Test build #53499 has finished for PR 11797 at commit c9284b8.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #53503 has finished for PR 11797 at commit 696ea5c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Merged to master

[dongjoon-hyun]

Oh, Thank you, @srowen !