[ci] Add Ray compatibility check informational CI job

[jeffreywang-anyscale]

Purpose

Ray installs vLLM via pip install 'vllm[audio]' constrained by its lock files. When a vLLM PR bumps or tightens a dependency (e.g. protobuf>=5.29.6), it can silently break Ray's ability to install vLLM in its environment. Today these conflicts are only discovered when the Ray team tries to upgrade, potentially blocking release timelines.

This PR adds a non-blocking CI job that runs pip install --dry-run of the built vLLM wheel against two Ray lock files (ray_py311_cu128.lock and rayllm_test_py311_cu128.lock). On conflict it surfaces a Buildkite annotation and sends a notification to Anyscale's internal slack channel. The job uses soft_fail: true so it never blocks the pipeline.

RFC: #33599

Test Plan & Result

• Verified locally that the check correctly detects the current protobuf>=5.29.6 vs protobuf==5.29.5 conflict against both lock files.
• Confirmed that the vllm== pin in rayllm_test_py311_cu128.lock is stripped to avoid false positives

TODO

• Require vLLM buildkite admin to add RAY_COMPAT_SLACK_WEBHOOK_URL as a pipeline secret after the PR merges.
• Create slack incoming webhook. AI: @jeffreywang-anyscale

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

The pull request introduces a useful CI job to proactively detect dependency conflicts between vLLM and Ray. This will help prevent integration issues that are currently only discovered during Ray's upgrade process. The implementation uses a non-blocking approach with Buildkite annotations and Slack notifications, which is appropriate for an informational check. I have identified a few high-severity issues related to the robustness of the bash script, specifically regarding error handling and safe JSON construction for notifications.

[gemini-code-assist]

The script should use set -e to ensure it terminates immediately if any setup command fails (e.g., curl failing to download a lock file or sed failing to process it). Without set -e, the script might continue with incomplete data, potentially leading to false positives where the compatibility check appears to pass because the constraints file was empty or missing.

Suggested change

	set -o pipefail
	set -eo pipefail

[gemini-code-assist]

Constructing a JSON payload by manually expanding environment variables inside a double-quoted string is fragile and insecure. If variables like BUILDKITE_BRANCH contain double quotes or other special characters, the resulting JSON will be malformed, causing the Slack notification to fail. It is safer to use a tool like jq or a small Python snippet to generate the JSON payload correctly.

    # Construct JSON payload safely using Python to avoid malformed JSON from special characters
    PAYLOAD=$(python3 -c '
import json, sys, os
failed = sys.argv[1]
pr = os.getenv("BUILDKITE_PULL_REQUEST", "N/A")
branch = os.getenv("BUILDKITE_BRANCH", "unknown")
url = os.getenv("BUILDKITE_BUILD_URL", "#")
data = {
    "text": ":warning: Ray Dependency Compatibility Check Failed",
    "blocks": [{
        "type": "section",
        "text": {
            "type": "mrkdwn",
            "text": f"*:warning: Ray Dependency Compatibility Check Failed*\nPR #{pr} on branch `{branch}` introduces dependencies that conflict with Ray'\''s lock file(s): {failed}\n<{url}|View Build>"
        }
    }]
}
print(json.dumps(data))
' "${FAILED_LOCKS[*]}")

    curl -s -X POST "$RAY_COMPAT_SLACK_WEBHOOK_URL" \
        -H 'Content-type: application/json' \
        -d "$PAYLOAD"