Spark 2.1.0 mapr 29106 (apache#150)

* [SPARK-20922][CORE] Add whitelist of classes that can be deserialized by the launcher.

Blindly deserializing classes using Java serialization opens the code up to
issues in other libraries, since just deserializing data from a stream may
end up execution code (think readObject()).

Since the launcher protocol is pretty self-contained, there's just a handful
of classes it legitimately needs to deserialize, and they're in just two
packages, so add a filter that throws errors if classes from any other
package show up in the stream.

This also maintains backwards compatibility (the updated launcher code can
still communicate with the backend code in older Spark releases).

Tested with new and existing unit tests.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes apache#18166 from vanzin/SPARK-20922.

(cherry picked from commit 8efc6e9)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>

(cherry picked from commit 772a9b9)

* [SPARK-20922][CORE][HOTFIX] Don't use Java 8 lambdas in older branches.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes apache#18178 from vanzin/SPARK-20922-hotfix.

(cherry picked from commit 0b25a7d)

(cherry picked from commit cf96fdd)

Author: mgorbov

launcher/src/test/java/org/apache/spark/launcher/LauncherServerSuite.java

@@ -17,23 +17,22 @@
 
 package org.apache.spark.launcher;
 
+import org.junit.Test;
+
 import java.io.Closeable;
 import java.io.IOException;
 import java.io.ObjectInputStream;
 import java.net.InetAddress;
 import java.net.Socket;
-import java.net.SocketException;
 import java.util.Arrays;
 import java.util.List;
 import java.util.concurrent.BlockingQueue;
 import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.Semaphore;
 import java.util.concurrent.TimeUnit;
 
-import org.junit.Test;
-import static org.junit.Assert.*;
-
 import static org.apache.spark.launcher.LauncherProtocol.*;
+import static org.junit.Assert.*;
 
 public class LauncherServerSuite extends BaseSuite {