[KYUUBI #5680] Optimize Spark engine pod name generation

### _Why are the changes needed?_

Close #5680

**Background**:
1) In default case, when kyuubi submit spark sql engine on kubernetes, will generate spark driver pod name with "kyuubi-${app_name}-${engine_ref_id}-driver".
2) And app_name will be "kyuubi_${shareLevel}_${engineType}_${appUser}_${engineRefId}" if not set by user.
3) In result, we may get spark driver pod name with "kyuubi-kyuubi-${shareLevel}-${engineType}-${appUser}-${engineRefId}-${engineRefId}-driver".
4) We were hoping for a more concise and readable name, such as "kyuubi-${shareLevel}-${engineType}-${appUser}-${app_name}-${engineRefId}-driver".

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate
**1) Before modification：**
Left is unset `spark.app.name`, and right is set to “ocean”
![image](https://github.com/apache/kyuubi/assets/45907917/350aa09f-a9cb-4cc4-bdc6-71b085ba0824)

**2) Modify the code**
![image](https://github.com/apache/kyuubi/assets/45907917/f9cbef91-9e1a-482a-82bb-a78caf669438)

**3) Build module and get the jar**
![image](https://github.com/apache/kyuubi/assets/45907917/b2cf7595-dd54-46eb-bb79-354f3301499f)

**4) Build a new images**
![image](https://github.com/apache/kyuubi/assets/45907917/c0418f2d-dcde-4845-a568-84e920892359)

**5) After modification:**
![image](https://github.com/apache/kyuubi/assets/45907917/5db95d5b-2550-41b7-a11c-d8e4a6c0f09f)

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

### _Was this patch authored or co-authored using generative AI tooling?_

No

Closes #5695 from Ocean22/master.

Closes #5680

6dbac57 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/util/KubernetesUtils.scala
5ddc9f6 [no 会 English] Update KubernetesUtils.scala
b685b60 [no 会 English] Update KubernetesUtils.scala
6a64e17 [no 会 English] Update KubernetesUtils.scala
1335bbd [no 会 English] Update EngineRef.scala
f2af955 [no 会 English] Update KubernetesUtils.scala
891f172 [no 会 English] Update KubernetesUtils.scala
c20b755 [no 会 English] Update EngineRef.scala
09f3ed1 [no 会 English] Update EngineRef.scala
64bd2c1 [no 会 English] Update EngineRef.scala

Lead-authored-by: Ocean22 <1058853680@qq.com>
Co-authored-by: no 会 English <45907917+Ocean22@users.noreply.github.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>

Author: Ocean22

kyuubi-server/src/main/scala/org/apache/kyuubi/util/KubernetesUtils.scala

@@ -120,8 +120,8 @@ object KubernetesUtils extends Logging {
     opt2.foreach { _ => require(opt1.isEmpty, errMessage) }
   }
 
-  private def getResourceNamePrefix(appName: String, engineRefId: String): String = {
-    s"$appName-$engineRefId"
+  private def getResourceNamePrefix(appName: String, engineRefId: Option[String]): String = {
+    engineRefId.map(refId => s"$appName-$refId").getOrElse(appName)
       .trim
       .toLowerCase(Locale.ROOT)
       .replaceAll("[^a-z0-9\\-]", "-")
@@ -134,7 +134,16 @@ object KubernetesUtils extends Logging {
       appName: String,
       engineRefId: String,
       forciblyRewrite: Boolean): String = {
-    lazy val resolvedResourceName = s"kyuubi-${getResourceNamePrefix(appName, engineRefId)}-driver"
+    val resourceNamePrefix = if (appName.contains(engineRefId)) {
+      getResourceNamePrefix(appName, None)
+    } else {
+      getResourceNamePrefix(appName, Some(engineRefId))
+    }
+    val resolvedResourceName = if (resourceNamePrefix.startsWith("kyuubi-")) {
+      s"$resourceNamePrefix-driver"
+    } else {
+      s"kyuubi-$resourceNamePrefix-driver"
+    }
     if (forciblyRewrite || resolvedResourceName.length > DRIVER_POD_NAME_MAX_LENGTH) {
       s"kyuubi-$engineRefId-driver"
     } else {
@@ -146,7 +155,16 @@ object KubernetesUtils extends Logging {
       appName: String,
       engineRefId: String,
       forciblyRewrite: Boolean): String = {
-    val resolvedResourceName = s"kyuubi-${getResourceNamePrefix(appName, engineRefId)}"
+    val resourceNamePrefix = if (appName.contains(engineRefId)) {
+      getResourceNamePrefix(appName, None)
+    } else {
+      getResourceNamePrefix(appName, Some(engineRefId))
+    }
+    val resolvedResourceName = if (resourceNamePrefix.startsWith("kyuubi-")) {
+      s"$resourceNamePrefix"
+    } else {
+      s"kyuubi-$resourceNamePrefix"
+    }
     if (forciblyRewrite || resolvedResourceName.length > EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH) {
       s"kyuubi-$engineRefId"
     } else {