-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-33724][K8S] Add decom script as a configuration param #30694
[SPARK-33724][K8S] Add decom script as a configuration param #30694
Conversation
Test build #132517 has finished for PR 30694 at commit
|
Kubernetes integration test starting |
Kubernetes integration test status failure |
K8s IT fails with
It locally works.
|
SPARK-33727 is filed for the K8s IT failure. |
Thanks for filing the infra issue @dongjoon-hyun. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (except one minor indentation issue)
Feel free to merge after fixing the indentation.
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
Outdated
Show resolved
Hide resolved
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
Outdated
Show resolved
Hide resolved
1df978c
to
46ca366
Compare
Test build #132535 has finished for PR 30694 at commit
|
Kubernetes integration test starting |
Kubernetes integration test status failure |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM.
The failure is irrelevant to this PR because this PR doesn't change the value effectively.
Merged to master for Apache Spark 3.2.0.
Makes the location of the decommission script used in Kubernetes for graceful shutdown configurable. Some environments don't use the Spark image builder and instead mount the decompressed Spark distro. In those envs configuring the location of the decommissioning script is required. New configuration parameter. Existing decommissioning integration test. Closes apache#30694 from holdenk/SPARK-33724-allow-decommissioning-script-location-to-be-configured. Authored-by: Holden Karau <hkarau@apple.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
### What changes were proposed in this pull request? Makes the location of the decommission script used in Kubernetes for graceful shutdown configurable. ### Why are the changes needed? Some environments don't use the Spark image builder and instead mount the decompressed Spark distro. In those envs configuring the location of the decommissioning script is required. ### Does this PR introduce _any_ user-facing change? New configuration parameter. ### How was this patch tested? Existing decommissioning integration test. Closes apache#30694 from holdenk/SPARK-33724-allow-decommissioning-script-location-to-be-configured. Authored-by: Holden Karau <hkarau@apple.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
What changes were proposed in this pull request?
Makes the location of the decommission script used in Kubernetes for graceful shutdown configurable.
Why are the changes needed?
Some environments don't use the Spark image builder and instead mount the decompressed Spark distro. In those envs configuring the location of the decommissioning script is required.
Does this PR introduce any user-facing change?
New configuration parameter.
How was this patch tested?
Existing decommissioning integration test.