[SPARK-35319][K8S][BUILD] Upgrade K8s client to 5.3.1

### What changes were proposed in this pull request?

This PR aims to upgrade K8s client to 5.3.1.

### Why are the changes needed?

This will bring the latest bug fixes.
- https://github.com/fabric8io/kubernetes-client/releases/tag/v5.3.1

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

K8s IT is manually tested like the following.

```
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local storage
- Launcher client dependencies
- SPARK-33615: Launcher client archives
- SPARK-33748: Launcher python client respecting PYSPARK_PYTHON
- SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python
- Launcher python client dependencies using a zip file
- Test basic decommissioning
- Test basic decommissioning with shuffle cleanup
- Test decommissioning with dynamic allocation & shuffle cleanups
- Test decommissioning timeouts
- Run SparkR on simple dataframe.R example
Run completed in 18 minutes, 33 seconds.
Total number of tests run: 27
Suites: completed 2, aborted 0
Tests: succeeded 27, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.2.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [  3.959 s]
[INFO] Spark Project Tags ................................. SUCCESS [  7.830 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  3.457 s]
[INFO] Spark Project Networking ........................... SUCCESS [  5.496 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  3.239 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  9.006 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  2.422 s]
[INFO] Spark Project Core ................................. SUCCESS [02:17 min]
[INFO] Spark Project Kubernetes Integration Tests ......... SUCCESS [21:05 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  23:59 min
[INFO] Finished at: 2021-05-05T11:59:19-07:00
[INFO] ------------------------------------------------------------------------
```

Closes #32443 from dongjoon-hyun/SPARK-35319.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

Author: dongjoon-hyun

dev/deps/spark-deps-hadoop-2.7-hive-2.3

@@ -155,26 +155,26 @@ jsr305/3.0.0//jsr305-3.0.0.jar
 jta/1.1//jta-1.1.jar
 jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
 kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
-kubernetes-client/5.3.0//kubernetes-client-5.3.0.jar
-kubernetes-model-admissionregistration/5.3.0//kubernetes-model-admissionregistration-5.3.0.jar
-kubernetes-model-apiextensions/5.3.0//kubernetes-model-apiextensions-5.3.0.jar
-kubernetes-model-apps/5.3.0//kubernetes-model-apps-5.3.0.jar
-kubernetes-model-autoscaling/5.3.0//kubernetes-model-autoscaling-5.3.0.jar
-kubernetes-model-batch/5.3.0//kubernetes-model-batch-5.3.0.jar
-kubernetes-model-certificates/5.3.0//kubernetes-model-certificates-5.3.0.jar
-kubernetes-model-common/5.3.0//kubernetes-model-common-5.3.0.jar
-kubernetes-model-coordination/5.3.0//kubernetes-model-coordination-5.3.0.jar
-kubernetes-model-core/5.3.0//kubernetes-model-core-5.3.0.jar
-kubernetes-model-discovery/5.3.0//kubernetes-model-discovery-5.3.0.jar
-kubernetes-model-events/5.3.0//kubernetes-model-events-5.3.0.jar
-kubernetes-model-extensions/5.3.0//kubernetes-model-extensions-5.3.0.jar
-kubernetes-model-metrics/5.3.0//kubernetes-model-metrics-5.3.0.jar
-kubernetes-model-networking/5.3.0//kubernetes-model-networking-5.3.0.jar
-kubernetes-model-node/5.3.0//kubernetes-model-node-5.3.0.jar
-kubernetes-model-policy/5.3.0//kubernetes-model-policy-5.3.0.jar
-kubernetes-model-rbac/5.3.0//kubernetes-model-rbac-5.3.0.jar
-kubernetes-model-scheduling/5.3.0//kubernetes-model-scheduling-5.3.0.jar
-kubernetes-model-storageclass/5.3.0//kubernetes-model-storageclass-5.3.0.jar
+kubernetes-client/5.3.1//kubernetes-client-5.3.1.jar
+kubernetes-model-admissionregistration/5.3.1//kubernetes-model-admissionregistration-5.3.1.jar
+kubernetes-model-apiextensions/5.3.1//kubernetes-model-apiextensions-5.3.1.jar
+kubernetes-model-apps/5.3.1//kubernetes-model-apps-5.3.1.jar
+kubernetes-model-autoscaling/5.3.1//kubernetes-model-autoscaling-5.3.1.jar
+kubernetes-model-batch/5.3.1//kubernetes-model-batch-5.3.1.jar
+kubernetes-model-certificates/5.3.1//kubernetes-model-certificates-5.3.1.jar
+kubernetes-model-common/5.3.1//kubernetes-model-common-5.3.1.jar
+kubernetes-model-coordination/5.3.1//kubernetes-model-coordination-5.3.1.jar
+kubernetes-model-core/5.3.1//kubernetes-model-core-5.3.1.jar
+kubernetes-model-discovery/5.3.1//kubernetes-model-discovery-5.3.1.jar
+kubernetes-model-events/5.3.1//kubernetes-model-events-5.3.1.jar
+kubernetes-model-extensions/5.3.1//kubernetes-model-extensions-5.3.1.jar
+kubernetes-model-metrics/5.3.1//kubernetes-model-metrics-5.3.1.jar
+kubernetes-model-networking/5.3.1//kubernetes-model-networking-5.3.1.jar
+kubernetes-model-node/5.3.1//kubernetes-model-node-5.3.1.jar
+kubernetes-model-policy/5.3.1//kubernetes-model-policy-5.3.1.jar
+kubernetes-model-rbac/5.3.1//kubernetes-model-rbac-5.3.1.jar
+kubernetes-model-scheduling/5.3.1//kubernetes-model-scheduling-5.3.1.jar
+kubernetes-model-storageclass/5.3.1//kubernetes-model-storageclass-5.3.1.jar
 lapack/1.3.2//lapack-1.3.2.jar
 leveldbjni-all/1.8//leveldbjni-all-1.8.jar
 libfb303/0.9.3//libfb303-0.9.3.jar

dev/deps/spark-deps-hadoop-3.2-hive-2.3

@@ -126,26 +126,26 @@ jsr305/3.0.0//jsr305-3.0.0.jar
 jta/1.1//jta-1.1.jar
 jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
 kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
-kubernetes-client/5.3.0//kubernetes-client-5.3.0.jar
-kubernetes-model-admissionregistration/5.3.0//kubernetes-model-admissionregistration-5.3.0.jar
-kubernetes-model-apiextensions/5.3.0//kubernetes-model-apiextensions-5.3.0.jar
-kubernetes-model-apps/5.3.0//kubernetes-model-apps-5.3.0.jar
-kubernetes-model-autoscaling/5.3.0//kubernetes-model-autoscaling-5.3.0.jar
-kubernetes-model-batch/5.3.0//kubernetes-model-batch-5.3.0.jar
-kubernetes-model-certificates/5.3.0//kubernetes-model-certificates-5.3.0.jar
-kubernetes-model-common/5.3.0//kubernetes-model-common-5.3.0.jar
-kubernetes-model-coordination/5.3.0//kubernetes-model-coordination-5.3.0.jar
-kubernetes-model-core/5.3.0//kubernetes-model-core-5.3.0.jar
-kubernetes-model-discovery/5.3.0//kubernetes-model-discovery-5.3.0.jar
-kubernetes-model-events/5.3.0//kubernetes-model-events-5.3.0.jar
-kubernetes-model-extensions/5.3.0//kubernetes-model-extensions-5.3.0.jar
-kubernetes-model-metrics/5.3.0//kubernetes-model-metrics-5.3.0.jar
-kubernetes-model-networking/5.3.0//kubernetes-model-networking-5.3.0.jar
-kubernetes-model-node/5.3.0//kubernetes-model-node-5.3.0.jar
-kubernetes-model-policy/5.3.0//kubernetes-model-policy-5.3.0.jar
-kubernetes-model-rbac/5.3.0//kubernetes-model-rbac-5.3.0.jar
-kubernetes-model-scheduling/5.3.0//kubernetes-model-scheduling-5.3.0.jar
-kubernetes-model-storageclass/5.3.0//kubernetes-model-storageclass-5.3.0.jar
+kubernetes-client/5.3.1//kubernetes-client-5.3.1.jar
+kubernetes-model-admissionregistration/5.3.1//kubernetes-model-admissionregistration-5.3.1.jar
+kubernetes-model-apiextensions/5.3.1//kubernetes-model-apiextensions-5.3.1.jar
+kubernetes-model-apps/5.3.1//kubernetes-model-apps-5.3.1.jar
+kubernetes-model-autoscaling/5.3.1//kubernetes-model-autoscaling-5.3.1.jar
+kubernetes-model-batch/5.3.1//kubernetes-model-batch-5.3.1.jar
+kubernetes-model-certificates/5.3.1//kubernetes-model-certificates-5.3.1.jar
+kubernetes-model-common/5.3.1//kubernetes-model-common-5.3.1.jar
+kubernetes-model-coordination/5.3.1//kubernetes-model-coordination-5.3.1.jar
+kubernetes-model-core/5.3.1//kubernetes-model-core-5.3.1.jar
+kubernetes-model-discovery/5.3.1//kubernetes-model-discovery-5.3.1.jar
+kubernetes-model-events/5.3.1//kubernetes-model-events-5.3.1.jar
+kubernetes-model-extensions/5.3.1//kubernetes-model-extensions-5.3.1.jar
+kubernetes-model-metrics/5.3.1//kubernetes-model-metrics-5.3.1.jar
+kubernetes-model-networking/5.3.1//kubernetes-model-networking-5.3.1.jar
+kubernetes-model-node/5.3.1//kubernetes-model-node-5.3.1.jar
+kubernetes-model-policy/5.3.1//kubernetes-model-policy-5.3.1.jar
+kubernetes-model-rbac/5.3.1//kubernetes-model-rbac-5.3.1.jar
+kubernetes-model-scheduling/5.3.1//kubernetes-model-scheduling-5.3.1.jar
+kubernetes-model-storageclass/5.3.1//kubernetes-model-storageclass-5.3.1.jar
 lapack/1.3.2//lapack-1.3.2.jar
 leveldbjni-all/1.8//leveldbjni-all-1.8.jar
 libfb303/0.9.3//libfb303-0.9.3.jar

resource-managers/kubernetes/core/pom.xml

@@ -30,7 +30,7 @@
   <properties>
     <sbt.project.name>kubernetes</sbt.project.name>
     <!-- Note: Please update the kubernetes client version in kubernetes/integration-tests/pom.xml -->
-    <kubernetes.client.version>5.3.0</kubernetes.client.version>
+    <kubernetes.client.version>5.3.1</kubernetes.client.version>
   </properties>
 
   <dependencies>

resource-managers/kubernetes/integration-tests/pom.xml

@@ -28,7 +28,7 @@
   <properties>
     <download-maven-plugin.version>1.3.0</download-maven-plugin.version>
     <extraScalaTestArgs></extraScalaTestArgs>
-    <kubernetes-client.version>5.3.0</kubernetes-client.version>
+    <kubernetes-client.version>5.3.1</kubernetes-client.version>
     <sbt.project.name>kubernetes-integration-tests</sbt.project.name>
 
     <!-- Integration Test Configuration Properties -->