[RayService][Bug] Serve Service May Select Pods That Are Actually Unready for Serving Traffic

[Yicheng-Lu-llll]

Why are these changes needed?

In short, when using Rayservice, the Serve Service may select the head Pod even if the serve app has not been submitted or is not running. This PR fixes it.

Detailed Reason:

In order to support "Raycluster with serve”, the Serving logic is now below:

1. Serve Service is created and selects only the Pods that have the label ray.io/serve=true.
2. The Raycluster controller creates Ray Pods with the label ray.io/serve=true regardless.
3. The Rayservice controller continuously checks if head Pod has a healthy HTTP proxy to serve traffic and manage ray.io/serve accordingly. As for worker Pods, their traffic readiness is determined by the readiness probe. See PR 1808 for more details.

As shown above , the Raycluster controller sets the label ray.io/serve=true regardless in order to support "Raycluster with serve". This causes the Serve Service to select the head Pod even if there is no running serve app or the HTTP proxy is not healthy. It will only get fixed after the Rayservice controller detects and changes the label to ray.io/serve=false.

Fix:

This PR set ray.io/serve=false for head Pod when creating to avoid the problem.

For people interested in "Raycluster with serve":

This PR removes the ray.io/serve=true selection label for the Serve Service created by 'ray cluster with serve'. This does not harm its functionality. As described in #1672, it exclusively uses Raycluster for serving. However, as mentioned above (see also PR 1808 for a better understanding), it is the Rayservice controller that checks the HTTP Proxy's health for Head Pod. Therefore, the 'Raycluster with serve' does not actually offer high availability. Even if an HTTP Proxy is unhealthy in the head Pod, the label ray.io/serve will not change to false, so, there is no need to have this selection label in this case.

Related issue number

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[kevin85421]

Discuss offline: Removing the label would result in the worker Pods lacking this label, preventing them from attaching to the K8s service managed by the RayService.

[Yicheng-Lu-llll]

Thank you for pointing out! I have correct it in 6de9391. Below is the test:

kubectl apply -f /home/ubuntu/kuberay/ray-operator/config/samples/ray_v1alpha1_rayservice.yaml
kubectl describe rayservice | grep "Service Status"
# Service Status:  Running
kubectl describe svc rayservice-sample-serve-svc
# Endpoints:         10.244.0.6:8000,10.244.0.7:8000
kubectl describe $(kubectl get pods -o=name | grep worker) | grep "ray.io/serve"
# ray.io/serve=true
kubectl describe $(kubectl get pods -o=name | grep head) | grep "ray.io/serve"
# ray.io/serve=true