Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci][core] Python 311 core release tests #38511

Merged
merged 16 commits into from
Aug 23, 2023
Merged

[ci][core] Python 311 core release tests #38511

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e1e2f26
Add a python 311 test
can-anyscale Jul 7, 2023
f54206e
Merge branch 'master' of https://github.com/ray-project/ray into can-…
rickyyx Aug 15, 2023
11bd3c1
updated
rickyyx Aug 15, 2023
7defb5e
update
rickyyx Aug 15, 2023
a306ef4
update
rickyyx Aug 16, 2023
1fed941
Merge branch 'pr-fix-job-311' into pr-py311-release-tests
rickyyx Aug 16, 2023
4607e6f
Merge branch 'master' of https://github.com/ray-project/ray into pr-p…
rickyyx Aug 16, 2023
a180606
update
rickyyx Aug 16, 2023
7d9fc9b
comments
rickyyx Aug 16, 2023
080a380
todo
rickyyx Aug 16, 2023
b327fcd
update
rickyyx Aug 16, 2023
f1d07d3
update
rickyyx Aug 16, 2023
c30b820
update
rickyyx Aug 16, 2023
45b83c6
Merge branch 'master' into pr-py311-core-release-tests
rickyyx Aug 17, 2023
e49f10c
Merge branch 'master' of github.com:ray-project/ray into pr-py311-cor…
rickyyx Aug 21, 2023
176c734
update
rickyyx Aug 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add a python 311 test 
Signed-off-by: can <can@anyscale.com>
  • Loading branch information
@can-anyscale
can-anyscale committed Jul 19, 2023
commit e1e2f261dbc4101f14edd5c123c3f4c2f7bc754d
10 changes: 10 additions & 0 deletions release/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,16 @@ compile_pip_requirements(
visibility = ["//visibility:private"],
)

compile_pip_requirements(
name = "requirements_byod_3.11",
requirements_in = "ray_release/byod/requirements_byod_3.11.in",
requirements_txt = "ray_release/byod/requirements_byod_3.11.txt",
tags = [
"team:ci",
],
visibility = ["//visibility:private"],
)

compile_pip_requirements(
name = "requirements_ml_byod_3.8",
requirements_in = "ray_release/byod/requirements_ml_byod_3.8.in",
Expand Down
1 change: 0 additions & 1 deletion release/ray_release/byod/byod.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,4 @@ sudo cp /tmp/wrk/wrk /usr/local/bin/wrk
EOF

COPY "$PIP_REQUIREMENTS" .
RUN "$HOME"/anaconda3/bin/pip install --no-cache-dir https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
RUN "$HOME"/anaconda3/bin/pip install --no-cache-dir -r "${PIP_REQUIREMENTS}"
1 change: 1 addition & 0 deletions release/ray_release/byod/byod_agent_stress_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,6 @@

set -exo pipefail

pip install https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
1 change: 1 addition & 0 deletions release/ray_release/byod/byod_rllib_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

set -exo pipefail

pip3 install https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
git clone https://github.com/ray-project/rl-experiments.git
unzip rl-experiments/halfcheetah-sac/2022-12-17/halfcheetah_1500_mean_reward_sac.zip -d ~/.
pip3 install torch==2.0.0+cu118 torchvision==0.15.1+cu118 --index-url https://download.pytorch.org/whl/cu118
Expand Down
1 change: 1 addition & 0 deletions release/ray_release/byod/requirements_byod_3.11.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Python requirements to run release tests from anyscale byod (cpu type)
6 changes: 6 additions & 0 deletions release/ray_release/byod/requirements_byod_3.11.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# This file is autogenerated by pip-compile with python 3.11
# To update, run:
#
# bazel run //release:requirements_byod_3.11.update
#
3 changes: 2 additions & 1 deletion release/ray_release/schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@
"enum": [
"3.7",
"3.8",
"3.9"
"3.9",
"3.11"
]
},
"frequency": {
Expand Down
36 changes: 35 additions & 1 deletion release/release_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2343,6 +2343,8 @@
team: core
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
- LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/ray/.mujoco/mujoco210/bin
Expand Down Expand Up @@ -2471,6 +2473,8 @@
team: core
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2511,6 +2515,8 @@
team: core
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2599,6 +2605,8 @@
team: core
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2639,6 +2647,8 @@
team: core
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2679,6 +2689,8 @@
team: core
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2762,6 +2774,8 @@
python: "3.8"
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2804,6 +2818,8 @@
python: "3.8"
cluster:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -2845,7 +2861,9 @@
team: serve
python: "3.8"
cluster:
byod:
byod:
pip:
- https://ray-ci-deps-wheels.s3.us-west-2.amazonaws.com/AutoROM.accept_rom_license-0.5.4-py3-none-any.whl
runtime_env:
- RLLIB_TEST_NO_JAX_IMPORT=1
cluster_env: app_config.yaml
Expand Down Expand Up @@ -5091,6 +5109,22 @@
cluster_env: app_config.yaml
cluster_compute: tpl_64_gce.yaml

- name: microbenchmark_311
group: core-daily-test
team: core
frequency: manual
working_dir: microbenchmark

python: "3.11"

cluster:
byod: {}
cluster_env: app_config.yaml
cluster_compute: tpl_64.yaml

run:
timeout: 1800
script: OMP_NUM_THREADS=64 RAY_ADDRESS=local python run_microbenchmark.py

- name: benchmark_worker_startup
group: core-daily-test
Expand Down