[SPARK-5692] [MLlib] Word2Vec save/load

[MechCoder]

Word2Vec model now supports saving and loading.

a] The Metadata stored in JSON format consists of "version", "classname", "vectorSize" and "numWords"
b] The data stored in Parquet file format consists of an Array of rows with each row consisting of 2 columns, first being the word: String and the second, an Array of Floats.

[MechCoder]

ping @mengxr @jkbradley

[SparkQA]

Test build #29473 has started for PR 5291 at commit d17cd8c.

[SparkQA]

Test build #29473 has finished for PR 5291 at commit d17cd8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Data(word: String, vector: Array[Float])
• This patch does not change any dependencies.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29473/
Test FAILed.

[SparkQA]

Test build #29476 has started for PR 5291 at commit bfe4c39.

[SparkQA]

Test build #29476 has finished for PR 5291 at commit bfe4c39.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Data(word: String, vector: Array[Float])
  • case class CreateStruct(children: Seq[NamedExpression]) extends Expression
• This patch does not change any dependencies.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29476/
Test PASSed.

[mengxr]

The ordering of keys are not guaranteed. We could test map equals directly.

assert(sameModel.getVectors.mapValues(_.toSeq) === model.getVectors.mapValues(_.toSeq))

[MechCoder]

I see. Why does this not work for a raw array?

[jkbradley]

@MechCoder Could you please add a quick description to the PR for record-keeping? Thanks!

[SparkQA]

Test build #29495 has started for PR 5291 at commit 1142f3a.

[MechCoder]

@mengxr I have fixed your comments and @jkbradley I have updated the PR description.

[SparkQA]

Test build #29495 has finished for PR 5291 at commit 1142f3a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Data(word: String, vector: Array[Float])
• This patch does not change any dependencies.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29495/
Test PASSed.

[mengxr]

LGTM. Merged into master. Thanks!