[ci][core] Python 311 core release tests

[rickyyx]

Why are these changes needed?

We will run python 3.11 release tests on the below core test to test compatibility:

1. weekly if there's no smoke test
2. nightly-3x if there's smoke test

Python 3.11 variance tests from core

• microbenchmark
• dask_on_ray_100gb_sort
• stress_test_state_api_scale

Other tests

• decision_tree_autoscaling_20_runs
• many_pgs_smoke_test
• stress_test_many_runtime_envs
• chaos_many_tasks_no_object_store
• chaos_many_actors
• chaos_dataset_shuffle_push_based_sort_1tb
• chaos_dataset_shuffle_sort_1tb
• chaos_dataset_shuffle_random_shuffle_1tb
• chaos_dataset_shuffle_push_based_random_shuffle_1tb
• agent_stress_test
• long_running_actor_deaths
• long_running_many_actors
• long_running_many_drivers
• long_running_many_tasks
• long_running_many_tasks_serialized_ids
• long_running_node_failures
• shuffle_100gb
• stress_test_placement_group
• autoscaling_shuffle_1tb_1000_partitions
• shuffle_20gb_with_state_api
• single_node_oom
• tune_air_oom
• benchmark_worker_startup
• dask_on_ray_1tb_sort
• many_nodes_actor_test_on_v2

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[can-anyscale]

Maybe run them first on PR

[rickyyx]

why is there no release build job on the pr?

[can-anyscale]

github webhook (which is what triggered jobs on buildkite) seems to be flaky in the last few days. Maybe rebase/merge with master again to trigger it.

[rickyyx]

Some tests could not be run (for reasons not related to this), resizing the set to include only the 3 tests:

• microbenchmark
• dask_on_ray_100gb_sort
• stress_test_state_api_scale

^ These should also cover a good number of core APIs.

Successful runs on PR: https://buildkite.com/ray-project/release-tests-pr/builds/49955#_