Skip to content

[SPARK-32067][K8S] Use unique ConfigMap name for executor pod template #29934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[SPARK-32067][K8S] Use unique ConfigMap name for executor pod template #29934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Constants.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ private[spark] object Constants {
val EXECUTOR_POD_SPEC_TEMPLATE_FILE_NAME = "pod-spec-template.yml"
val EXECUTOR_POD_SPEC_TEMPLATE_MOUNTPATH = "/opt/spark/pod-template"
val POD_TEMPLATE_VOLUME = "pod-template-volume"
val POD_TEMPLATE_CONFIGMAP = "podspec-configmap"
val POD_TEMPLATE_CONFIGMAP = "driver-podspec-conf-map"
val POD_TEMPLATE_KEY = "podspec-configmap-key"

// Miscellaneous
Expand Down
6 changes: 4 additions & 2 deletions ...s/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,16 @@ private[spark] class PodTemplateConfigMapStep(conf: KubernetesConf)

private val hasTemplate = conf.contains(KUBERNETES_EXECUTOR_PODTEMPLATE_FILE)

private val configmapName = s"${conf.resourceNamePrefix}-$POD_TEMPLATE_CONFIGMAP"
Copy link
Member

@dongjoon-hyun dongjoon-hyun Oct 5, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the following status.

tpcds-py-eebc4474fa60b6f7-driver-conf-map      1      5s
tpcds-py-eebc4474fa60b6f7-podspec-configmap    1      5s

This is inconsistent because Driver is using driver-conf-map postfix and executor is using podspec-configmap.

resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala:    val configMapName = s"${conf.resourceNamePrefix}-driver-conf-map"

Let's use -exec-conf-map as a postfix.

Copy link
Contributor Author

@stijndehaes stijndehaes Oct 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wel actually this configmap is not used by the executors but also by the driver. The driver needs this configmap to spawn executors with the right template. So I'll change the name to:driver-podspec-conf-map. Let me know if that is fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you share your result from k get cm? Does it looks reasonably consistent?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, please update your PR description. I added AFTER paragraph at this PR description. You can fill in the final output there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During running the integration tests when watching the configmap it looks something like this:

aaece65ef82e4a30b7b7800aad600d4f   spark-test-app-aac9f37502b2ca55-driver-conf-map   1      0s
aaece65ef82e4a30b7b7800aad600d4f   spark-test-app-aac9f37502b2ca55-driver-podspec-conf-map   1      0

For me this looks nice because it's clear that the confimap is used by the driver. And when sorted alphabetically they are neatly together.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Thank you, @stijndehaes .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!


def configurePod(pod: SparkPod): SparkPod = {
if (hasTemplate) {
val podWithVolume = new PodBuilder(pod.pod)
.editSpec()
.addNewVolume()
.withName(POD_TEMPLATE_VOLUME)
.withNewConfigMap()
.withName(POD_TEMPLATE_CONFIGMAP)
.withName(configmapName)
.addNewItem()
.withKey(POD_TEMPLATE_KEY)
.withPath(EXECUTOR_POD_SPEC_TEMPLATE_FILE_NAME)
Expand Down Expand Up @@ -76,7 +78,7 @@ private[spark] class PodTemplateConfigMapStep(conf: KubernetesConf)
val podTemplateString = Files.toString(new File(podTemplateFile), StandardCharsets.UTF_8)
Seq(new ConfigMapBuilder()
.withNewMetadata()
.withName(POD_TEMPLATE_CONFIGMAP)
.withName(configmapName)
.endMetadata()
.addToData(POD_TEMPLATE_KEY, podTemplateString)
.build())
Expand Down
8 changes: 5 additions & 3 deletions ...e/src/test/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStepSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,14 @@
*/
package org.apache.spark.deploy.k8s.features

import java.io.{File, PrintWriter}
import java.io.PrintWriter
import java.nio.file.Files

import io.fabric8.kubernetes.api.model.ConfigMap

import org.apache.spark.{SparkConf, SparkFunSuite}
import org.apache.spark.deploy.k8s._
import org.apache.spark.deploy.k8s.Constants._
import org.apache.spark.util.Utils

class PodTemplateConfigMapStepSuite extends SparkFunSuite {
Expand Down Expand Up @@ -56,8 +57,9 @@ class PodTemplateConfigMapStepSuite extends SparkFunSuite {

assert(configuredPod.pod.getSpec.getVolumes.size() === 1)
val volume = configuredPod.pod.getSpec.getVolumes.get(0)
val generatedResourceName = s"${kubernetesConf.resourceNamePrefix}-$POD_TEMPLATE_CONFIGMAP"
assert(volume.getName === Constants.POD_TEMPLATE_VOLUME)
assert(volume.getConfigMap.getName === Constants.POD_TEMPLATE_CONFIGMAP)
assert(volume.getConfigMap.getName === generatedResourceName)
assert(volume.getConfigMap.getItems.size() === 1)
assert(volume.getConfigMap.getItems.get(0).getKey === Constants.POD_TEMPLATE_KEY)
assert(volume.getConfigMap.getItems.get(0).getPath ===
Expand All @@ -70,7 +72,7 @@ class PodTemplateConfigMapStepSuite extends SparkFunSuite {

val resources = step.getAdditionalKubernetesResources()
assert(resources.size === 1)
assert(resources.head.getMetadata.getName === Constants.POD_TEMPLATE_CONFIGMAP)
assert(resources.head.getMetadata.getName === generatedResourceName)
assert(resources.head.isInstanceOf[ConfigMap])
val configMap = resources.head.asInstanceOf[ConfigMap]
assert(configMap.getData.size() === 1)
Expand Down