KubernetesPodOperator: xcom sidecar can't terminate when share process namespace is set

[Laerte]

Apache Airflow Provider(s)

cncf-kubernetes

Versions of Apache Airflow Providers

apache-airflow-providers-cncf-kubernetes==8.3.4

Apache Airflow version

v2.9.1

Operating System

Debian GNU/Linux 11 (bullseye)

Deployment

Google Cloud Composer

Deployment details

No response

What happened

xcom sidecar container fail to terminate because it tries to kill pid 1:

airflow/providers/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py

Line 858 in 09d8a80

self._exec_pod_command(resp, "kill -2 1")

But fail:

[2024-12-18, 16:26:28 UTC] {pod_manager.py:725} INFO - Checking if xcom sidecar container is started.
[2024-12-18, 16:26:28 UTC] {pod_manager.py:728} INFO - The xcom sidecar container is started.
[2024-12-18, 16:26:29 UTC] {pod_manager.py:805} INFO - Running command... if [ -s /***/xcom/return.json ]; then cat /***/xcom/return.json; else echo __***_xcom_result_empty__; fi
[2024-12-18, 16:26:30 UTC] {pod_manager.py:805} INFO - Running command... kill -2 1
[2024-12-18, 16:26:30 UTC] {pod_manager.py:815} INFO - stderr from command: sh: can't kill pid 1: Permission denied
[2024-12-18, 16:26:30 UTC] {pod.py:586} INFO - xcom result: 
{"hello": "world"}

This happen because when we use share process namespace process will not run with PID 1.

What you think should happen instead

The xcom sidecar container should terminate gracefully, maybe we should switch the command to kill -9 $(pgrep -f trap) by default?

How to reproduce

Create a task using GKEStartPodOperator and set full_pod_spec on V1PodSpec set share_process_namespace to True

from kubernetes.client import models as k8s

GKEStartPodOperator(
        task_id="gke-task",
        do_xcom_push=True,
        full_pod_spec=k8s.V1Pod(
            spec=k8s.V1PodSpec(
                share_process_namespace=True,
                containers=[
                    k8s.V1Container(
                        name="container1",
                        image="alpine",
                        command=["sh", "-c"],
                        args=[
                            """mkdir -p /airflow/xcom && echo '{"Hello": "World"}' > /airflow/xcom/return.json && echo 'Done'""",
                        ],
                    ),
                    k8s.V1Container(
                        name="container2",
                        image="alpine",
                        command=["sh", "-c"],
                        args=["sleep 15"],
                    ),
                ],
            ),
        ),
    )

Anything else

Every time.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.